Datapipe Files

Real-time, incremental ETL library for ML with record-level depend

Status: Alpha

Brought to you by: otatarinova

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
README.md	2023-07-10	902 Bytes	0
v0.12.0-alpha.2 source code.tar.gz	2023-07-10	999.6 kB	0
v0.12.0-alpha.2 source code.zip	2023-07-10	1.0 MB	0
Totals: 3 Items		2.0 MB	0

WIP 0.12.0

Breaking changes

Move cli from datapipe-app to datapipe
Remove separate datapipe step status command, now it's a flag: datapipe step list --status
DatatableTransform moved from datapipe.compute to datapipe.core_steps

New features

Add datapipe.store.qdrant.QdrantStore
Add DatatableBatchTransform pipeline step

Refactorings

Add labels arg and property to ComputeStep base class
Add labels arg to BatchTransform and BatchTransformStep
Add labels arg to BatchGenerate and DatatableTransformStep
Add labels arg to UpdateExternalTable and DatatableTransformStep
Large refactoring, ComputeStep now contains pieces of overridable functions for run_full and run_changelist
Add prototype events logging for steps, add event_logger.log_step_full_complete, add table datapipe_step_events

Source: README.md, updated 2023-07-10

Other Useful Business Software

Deploy Apps in Seconds with Cloud Run Icon

Deploy Apps in Seconds with Cloud Run

Host and run your applications without the need to manage infrastructure. Scales up from and down to zero automatically.

Cloud Run is the fastest way to deploy containerized apps. Push your code in Go, Python, Node.js, Java, or any language and Cloud Run builds and deploys it automatically. Get fast autoscaling, pay only when your code runs, and skip the infrastructure headaches. Two million requests free per month. And new customers get $300 in free credit.

Try Cloud Run Free

$300 in Free Credit for Your Google Cloud Projects Icon

$300 in Free Credit for Your Google Cloud Projects

Build, test, and explore on Google Cloud with $300 in free credit. No hidden charges. No surprise bills.

Launch your next project with $300 in free Google Cloud credit—no hidden charges. Test, build, and deploy without risk. Use your credit across the Google Cloud platform to find what works best for your needs. After your credits are used, continue building with free monthly usage products. Only pay when you're ready to scale. Sign up in minutes and start exploring.

Start Free Trial

Recommended Projects

CloverDX
Design, automate, operate and publish data pipelines at scale
Pentaho
Pentaho offers comprehensive data integration and analytics platform.
rudderstack
Privacy and Security focused Segment-alternative, in Golang
Apache Kafka
Mirror of Apache Kafka
Conduit
Conduit streams data between data stores. Kafka Connect replacement