Download
Kedro is an open sourced Python framework for creating maintainable and modular data science code. Provides the scaffolding to build more complex data and machine-learning pipelines. In addition, there's a focus on spending less time on the tedious "plumbing" required to maintain data science code; this means that you have more time to solve new problems. Standardises team workflows; the modular structure of Kedro facilitates a higher level of collaboration when teams solve problems together. Makes a seamless transition from development to production, as you can write quick, throw-away exploratory code and transition to maintainable, easy-to-share, code experiments quickly. Puts the "engineering" back into data science because it borrows concepts from software engineering and applies them to machine-learning code. It is the foundation for clean, data science code.
Features
- Kedro's pipeline visualisation plugin shows a blueprint of your developing data and machine-learning workflows, provides data lineage
- Keeps track of machine-learning experiments and makes it easier to collaborate with business stakeholders
- A series of lightweight data connectors used to save and load data across many different file formats and file systems
- Supported file formats include Pandas, Spark, Dask, NetworkX, Pickle, Plotly, Matplotlib and many more
- The Data Catalog supports S3, GCP, Azure, sFTP, DBFS and local filesystems
- The Data Catalog also includes data and model snapshots for file-based systems
Project Samples
