Open Source Linux ETL Tools
Sort By:
ETL Tools for Linux
View 25 business solutionsBrowse free open source ETL tools and projects for Linux below. Use the toggles on the left to filter open source ETL tools by OS, license, language, programming language, and project status.
-
Orchestrate Your AI Agents with ZenflowZenflow orchestrates AI agents like a real engineering system. With parallel execution, spec-driven workflows, and deep multi-repo understanding, agents plan, implement, test, and verify end-to-end. Upgrade to AI workflows that work the way your team does.
-
Auth0 for AI Agents now in GAConnect your AI agents to apps and data more securely, give users control over the actions AI agents can perform and the data they can access, and enable human confirmation for critical agent actions.
-
1
Pentaho
Pentaho offers comprehensive data integration and analytics platform.
Pentaho couples data integration with business analytics in a modern platform to easily access, visualize and explore data that impacts business results. Use it as a full suite or as individual components that are accessible on-premise, in the cloud, or on-the-go (mobile). Pentaho enables IT and developers to access and integrate data from any source and deliver it to your applications all from within an intuitive and easy to use graphical tool. The Pentaho Enterprise Edition Free Trial can be obtained from https://pentaho.com/download/ -
2
Logstash
Centralize, transform and stash your data
Logstash is a server-side data processing pipeline that dynamically ingests data from numerous sources, transforms it, and ships it to your favorite “stash” regardless of format or complexity. It supports and ingests data of all shapes, sizes and sources, dynamically transforms and prepares this data, and transports it to the output of your choice. Logstash is extensible, with over 200 plugins available to let you create and configure your pipeline how you choose. -
3
Ethereum ETL
Python scripts for ETL (extract, transform and load) jobs for Ethereum
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery. Ethereum ETL lets you convert blockchain data into convenient formats like CSVs and relational databases. -
4
MyDBF2MySQL
Extract, transform, and load DBF into MySQL
This is an ETL software which loads data from DBF/XBase files into MySQL. This utility has command line interface, designed to work without user interaction. -
Grafana: The open and composable observability platformGrafana is the open source analytics & monitoring solution for every database.
-
5
The BEE Project is a suite of tools supporting Business Intelligence project implementation including ETL tool and OLAP server and a thin client. The ROLAP server ensures multipass SQL generation and powerful cache management (utilizes MySQL RDBMS).
-
6
SQL*Plus Commander
Text-based user interface to query data on Oracle DB in a smart way
SQL*Plus Commander is Text-based user interface (TUI) / framework to query data on Oracle DB in a smart way. It consists in a fully customizable script shell for bash and ksh. It executes custom queries or procedures on DB with SQLPlus for Oracle. The results of queries can be browsed in a colorful text interface resulting data from a query can be selected and passed dinamically as parameters for others queries or procedures It may be useful for people who runs frequently a limited number of query and uses the results as parameters for other queries. suggested for DBA activities, log tables browsing. downloaded version contains a demo with HR data model from oracle.com Try it and let me know if you find it useful any idea or suggestion will be appreciated -
7
CloverDX
Design, automate, operate and publish data pipelines at scale
Please, visit www.cloverdx.com for latest product versions. Data integration platform; can be used to transform/map/manipulate data in batch and near-realtime modes. Suppors various input/output formats (CSV,FIXLEN,Excel,XML,JSON,Parquet, Avro,EDI/X12,HL7,COBOL,LOTUS, etc.). Connects to RDBMS/JMS/Kafka/SOAP/Rest/LDAP/S3/HTTP/FTP/ZIP/TAR. CloverDX offers 100+ specialized components which can be further extended by creation of "macros" - subgraphs - and libraries, shareable with 3rd parties. Simple data manipulation jobs can be created visually. More complex business logic can be implemented using Clover's domain-specific-language CTL, in Java or languages like Python or JavaScript. Through its DataServices functionality, it allows to quickly turn data pipelines into REST API endpoints. The platform allows to easily scale your data job across multiple cores or nodes/machines. Supports Docker/Kubernetes deployments and offers AWS/Azure images in their respective marketplace -
8
PDI Data Vault framework
Data Vault loading automation using Pentaho Data Integration.
A metadata driven 'tool' to automate loading a designed Data Vault. It consists of a set of Pentaho Data Integration and database objects. Thel Virtual Machine (VMware) is a 64 bit Ubuntu Server 14.04, with MySQL (Percona Server) and PostgreSQL 9.4 as the database flavours and PDI version 5.2 CE. NB: Directory version_2.4 contains the most recent Virtual Machine. The readme.txt contains info about that VM. -
9
HPCC Systems
End-to-end big data in a massively scalable supercomputing platform.
HPCC Systems® (www.hpccsystems.com) from LexisNexis® Risk Solutions is a proven, open source solution for Big Data insights that can be implemented by businesses of all sizes. With HPCC Systems, developers can design applications with Big Data at their core, enabling businesses to better analyze and understand data at scale, improving business time to results and decisions. HPCC Systems offers a consistent data-centric programming language, two processing platforms and a single, complete end-to-end architecture for efficient processing. Read our blog (http://hpccsystems.com/blog ), or connect with us on Twitter (@hpccsystems), Facebook (https://www.facebook.com/hpccsystems ) and LinkedIn (http://www.linkedin.com/company/hpcc-systems) HPCC Systems is available on AWS & can be configured through the Instant Cloud Solution. -
Trumba is an All-in-one Calendar Management and Event Registration platformPublish, promote and track your events more affordably and effectively—all in one place.
-
10
GeoKettle is a powerful, metadata-driven spatial ETL (Extract, Transform and Load) tool dedicated to the integration of different data sources for building and updating geospatial databases, data warehouses and services.
-
11
Datapipe
Real-time, incremental ETL library for ML with record-level depend
Datapipe is a real-time, incremental ETL library for Python with record-level dependency tracking. Datapipe is designed to streamline the creation of data processing pipelines. It excels in scenarios where data is continuously changing, requiring pipelines to adapt and process only the modified data efficiently. This library tracks dependencies for each record in the pipeline, ensuring minimal and efficient data processing. -
12
Jaspersoft ETL is a data integration platform providing high performance data extract-transform-load (ETL) capabilities. Jaspersoft ETL is appropriate for all analytic and operational data integration needs. Activity on this project is located at jas
-
13
Diffs, patches, and revision control for CSV files, spreadsheets, and databases.
-
14
KETL(tm) is a production ready ETL platform. The engine is built upon an open, multi-threaded, XML-based architecture. KETL's is designed to assist in the development and deployment of data integration efforts which require ETL and scheduling
-
15
-
16
AWS Data Wrangler
Pandas on AWS, easy integration with Athena, Glue, Redshift, etc.
An AWS Professional Service open-source python initiative that extends the power of Pandas library to AWS connecting DataFrames and AWS data-related services. Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON, and EXCEL). Built on top of other open-source projects like Pandas, Apache Arrow and Boto3, it offers abstracted functions to execute usual ETL tasks like load/unload data from Data Lakes, Data Warehouses, and Databases. Convert the column name to be compatible with Amazon Athena and the AWS Glue Catalog. Run a query against AWS CloudWatchLogs Insights and convert the results to Pandas DataFrame. Get QuickSight dashboard ID given a name and fails if there is more than 1 ID associated with this name. List IAM policy assignments in the current Amazon QuickSight account. -
17
AvaSattva
Search replace files or pipe
See https://github.com/qualiu/msr/ Match/Search/Replace: msr.exe/msr-Win32.exe/msr.cygwin/msr.gcc**/msr-i386.gcc** Match/Search/Replace/Execute/* Files/Pipe Lines/Blocks. Filter/Load/Extract/Transform/Stats/* Files/Pipe Lines/Blocks. Not-IN-latter: nin.exe/nin-Win32.exe/nin.cygwin/nin.gcc**/nin-i386.gcc** Get Exclusive/Mutual Line-Set or Key-Set; Remove Line-Set or Key-Set matched in latter file/pipe; Get Unique/Mutual/Distribution/Stats/* Files/Pipe Line-Set or Key-Set. Match/Search/Replace files/pipe text with plain/Regex syntax. And for ETL alike work like Load and filter files -> Extract -> Transform output. For replacing files, you can preview and backup, in multiple directories and files or pipe, with plain text matching or using general Regex as C++, C#, Java, Scala; So msr is a good tool to learn and test Regex since it has different colors for matched groups captured by the Regex pattern. -
18
BIAutomationTool
Tool created to aggregate commands to disparate ETL tools
This project was created to allow executing ETL jobs/tasks from a single command line tool with the same syntax, no matter what tool you were executing in. As long as you have a command line client for the ETL tool, you can configure the BIAutomationTool to use it. -
19
Parse, analyze and -- most importantly -- use COBOL data definitions. This gives you access to COBOL data from Python programs. Write data analyzers, one-time data conversion utilities and Python programs that are part of COBOL systems. Really.
-
20
Civi Data Integration
This is a Pentaho Data Integration plugin for CiviCRM.
This is a Pentaho Data Integration plugin for CiviCRM. It allows you to take advantage of the power of Pentaho Data Integration tools and use it with your CiviCRM instance. -
21
DataStreamProcessor
TPL (transfer,parse,load) tool for batch files.
This is an enterprise-strength system for batch file processing, e.g. transfer, parse and load data using batch (text) files within and without the enterprise. The system is controlled through parameters and doesn't require any programming, code generation or code deployment. This is a heavy duty back end system with no GUI. Nonetheless it's very easy to use, easier than most GUI-based ETLs, even easier to install. It currently supports 4 major dbs: Oracle, Sybase, MySQL, MSSQL. The free community edition allows to process about 10 files a day depending on the setup. For support and licensing go to the www.datastreamprocessor.com -
22
Drop is a graphical interface for the GPL project Wets (hosted in SourceForge). Wets is an ETL software.
-
23
-
24
-
25
EplSite ETL
ETL Based on Perl With WEB Interface
EplSite ETL is a tool to do easy the data migrations, doing extraction, transformation, validation and load in a very fast way. It was built by people involved in data migrations so, it contains the necessary to do the migration(Extract Transformation, validation and load) and do it well.
