{"repo":"ryfeus/stepfunctions2processing","free":true,"listed":false,"github":"https://github.com/ryfeus/stepfunctions2processing","clone":"git clone https://github.com/ryfeus/stepfunctions2processing.git","description":"Configuration with AWS step functions and lambdas which initiates processing from activity state","language":"Python","stars":118,"topics":["aws","stepfunctions","lambda","serverless","workflow","processing","aws-fargate","aws-batch","serverless-processing","fargate"],"license":"MIT","category":"workflow-automation","readme_excerpt":"stepfunctions2processing Configuration with AWS step functions and lambdas which initiates processing from activity state Motivation Currently AWS Step Function service provides the best way to unite together serverless processing and cluster processing. Serverless processing can be extremely useful for the cases when you need to have a very good scalability or cheap processing. But for the long or CPU heavy processing you may still need to use cluster. The current version of step function expects cluster (except for ECS and Batch) to make requests to the Step Function activity. For clusters with extremely expensive machines it may not be the best way for handling so this project allows to transform activity state into active processing trigger. Examples here also include AWS Batch and AWS Fargate TL;DR : Project allows to initiate processing from AWS Step function instead of making state requests towards it. Examples AWS Lambda example Scheme 1. Step function process starts and input data is provided to lambda 2. Lambda processes data and produces json output which is then given to the activity. 3. Checker lambda makes regular requests to the activity in case activity is in \"ActivityScheduled\" state. 4. Activty returns token and input json. 5. Checker lambda initiates processing lambda which takes token and input json as input. 6. Processing lambda conducts processing part and return token and output json to activity. 7. Step functions provides output json from the activity ","default_branch":null,"files":null,"tree":[],"storefront":"/r/ryfeus","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ryfeus/stepfunctions2processing/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}