Prefect

Easiest way to automate your data: prefect.io. Followed tutorial

As a test, I am going to string together synapsemonitor and prefect to create a very small ETL pipeline.

1. Install packages

    conda create -n prefect python=3.7
    pip install -r requirements.txt
   

2. Synapse monitor

    synapsemonitor view syn24187217 --days 40
   

3. Stringing together synapsemonitor with prefect

    python demo.py
   

4. Showing dashboard

    prefect backend server
    prefect server start
    # visit http://localhost:8080/default
    prefect agent local start