Search Results for "high compression ratio"
Sort By:
Showing 187 open source projects for "high compression ratio"
View related business solutions-
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
1
Zstandard
Zstandard - Fast real-time compression algorithm
Zstandard is a fast compression algorithm, providing high compression ratios. It also offers a special mode for small data, called dictionary compression. The reference library offers a very wide range of speed / compression trade-off, and is backed by an extremely fast decoder (see benchmarks below). Zstandard library is provided as open source software using a BSD license. -
2
XZ Utils
Open-source compression utility and library
xz is a widely used open-source compression utility and library that implements the high-ratio LZMA and LZMA2 compression algorithms. It provides both command-line tools and a reusable C library, enabling developers and system administrators to compress and decompress files efficiently across many environments. The project is known for delivering strong compression performance while maintaining reasonable memory usage, making it suitable for software distribution, backups, and archival workflows. xz is commonly included in Unix-like operating systems and is often used as a standard packaging format in Linux ecosystems. ... -
3
LZ4
Extremely fast compression algorithm
LZ4 is lossless compression algorithm, providing compression speed > 500 MB/s per core (>0.15 Bytes/cycle). It features an extremely fast decoder, with speed in multiple GB/s per core (~1 Byte/cycle). A high compression derivative, called LZ4_HC, is available, trading customizable CPU time for compression ratio. LZ4 library is provided as open-source software using a BSD license. -
4
MCP Image Compression
A high-performance image compression microservice based on MCP
The MCP Image Compression server is a high-performance microservice based on the Model Context Protocol architecture. It focuses on providing fast and high-quality image compression capabilities to help developers optimize image resources for websites and applications, improving loading speed and user experience. -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
Wan2.2
Wan2.2: Open and Advanced Large-Scale Video Generative Model
...It introduces a Mixture-of-Experts (MoE) architecture that splits the denoising process across specialized expert models, increasing total model capacity without raising computational costs. Wan2.2 integrates meticulously curated cinematic aesthetic data, enabling precise control over lighting, composition, color tone, and more, for high-quality, customizable video styles. The model is trained on significantly larger datasets than its predecessor, greatly enhancing motion complexity, semantic understanding, and aesthetic diversity. Wan2.2 also open-sources a 5-billion parameter high-compression VAE-based hybrid text-image-to-video (TI2V) model that supports 720P video generation at 24fps on consumer-grade GPUs like the RTX 4090. ... -
6
Brotli
Brotli compression format
...This happens when "one-shot" decoding API is used (or input chunk for streaming API is not limited), input size (chunk size) is larger than 2GiB, and input contains uncompressed blocks. After the overflow happens, memcpy is invoked with a gigantic num value, that will likely cause the crash. Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression. The specification of the Brotli Compressed Data Format is defined in RFC 7932. ... -
7
WAL-G
Archival and Restoration for databases in the Cloud
...To configure the compression method used for backups. Possible options are lz4, lzma, zstd, brotli. The default method is lz4. LZ4 is the fastest method, but the compression ratio is bad. LZMA is way much slower. However, it compresses backups about 6 times better than LZ4. Brotli and zstd are a good trade-off between speed and compression ratio, which is about 3 times better than LZ4. -
8
DwarFS
A fast high compression read-only file system for Linux, Windows
The Deduplicating Warp-speed Advanced Read-only File System. A fast high compression read-only file system for Linux and Windows. DwarFS is a read-only file system with a focus on achieving very high compression ratios in particular for very redundant data. This probably doesn't sound very exciting, because if it's redundant, it should compress well. However, I found that other read-only, compressed file systems don't do a very good job at making use of this redundancy. ... -
9
compress
Optimized Go Compression Packages
This package provides various compression algorithms. zstandard compression and decompression in pure Go. S2 is a high-performance replacement for Snappy. Optimized deflate packages which can be used as a dropin replacement for gzip, zip and zlib. snappy is a drop-in replacement for github.com/golang/snappy offering better compression and concurrent streams. huff0 and FSE implementations for raw entropy encoding. gzhttp Provides client and server wrappers for handling gzipped requests efficiently. pgzip is a separate package that provides a very fast parallel gzip implementation. ... -
Application Monitoring That Won't Slow Your App DownFull APM with errors, performance, logs, and uptime monitoring. 99.999% uptime SLA on the platform itself.
-
10
FramePack
Lets make video diffusion practical
FramePack explores compact representations for sequences of image frames, targeting tasks where many near-duplicate frames carry redundant information. The idea is to “pack” frames by detecting shared structure and storing differences efficiently, which can accelerate training or inference on video-like data. By reducing I/O and memory bandwidth, datasets become lighter to load while models still see the essential temporal variation. The repository demonstrates both packing and unpacking... -
11
7-Zip
A free file archiver for extremely high compression
7-Zip is a file archiver with a high compression ratio. 7-Zip works in Windows 11 / 10 / 8 / 7 / Vista / XP / 2022 / 2019 / 2016 / 2012 / 2008 / 2003 / 2000. You can use 7-Zip on any computer, including a computer in a commercial organization. You don't need to register or pay for 7-Zip. And there is command line version for Linux and macOS. Most of the source code is under the GNU LGPL license. -
12
IoTDB
Apache IoTDB
Apache IoTDB (Database for Internet of Things) is an IoT native database with high performance for data management and analysis, deployable on the edge and the cloud. Due to its light-weight architecture, high performance and rich feature set together with its deep integration with Apache Hadoop, Spark and Flink, Apache IoTDB can meet the requirements of massive data storage, high-speed data ingestion and complex data analysis in the IoT industrial fields. In the scene of factories, there... -
13
Snappy
A fast compressor/decompressor
Snappy is a compression/decompression library. It does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. For instance, compared to the fastest mode of zlib, Snappy is an order of magnitude faster for most inputs, but the resulting compressed files are anywhere from 20% to 100% bigger. -
14
nghttp2
HTTP/2 C Library and tools
...Since then we have updated nghttp2 library constantly to the latest specification and nghttp2 is now one of the most mature HTTP/2 implementations. HTTP/2 utilizes header compression method called HPACK. We offer HPACK encoder and decoder are available as public API. nghttp2 library itself is a bit low-level. The experimental high-level C++ API is also available. We have Python binding of this library, but we have not covered everything yet. -
15
OrioleDB
Building a modern cloud-native storage engine
OrioleDB is a next-generation storage engine for PostgreSQL, aiming to enhance scalability, performance, and modern data management. Designed as a table access method (TAM) extension, it introduces features like MVCC improvements, compression, and efficient concurrency handling. OrioleDB seeks to future-proof PostgreSQL by enabling cloud-native, high-throughput workloads without altering the core PostgreSQL behavior. -
16
DeepSpeed
Deep learning optimization library: makes distributed training easy
...Achieve excellent system throughput and efficiently scale to thousands of GPUs 3. Train/Inference on resource constrained GPU systems 4. Achieve unprecedented low latency and high throughput for inference 5. Achieve extreme compression for an unparalleled inference latency and model size reduction with low costs DeepSpeed offers a confluence of system innovations, that has made large scale DL training effective, and efficient, greatly improved ease of use, and redefined the DL training landscape in terms of scale that is possible. ... -
17
HDF5
Official HDF5® Library Repository
HDF5 (Hierarchical Data Format v5) is a widely-used data management library and file format for storing large and complex scientific data sets efficiently. -
18
IOWOW
A C utility library and persistent key/value storage engine
IOWOW is a lightweight, high-performance key-value store and data storage library written in C, designed for use in embedded systems and high-speed applications that require efficient file-based storage. -
19
Audiogen Codec
48khz stereo neural audio codec for general audio
AGC (Audiogen Codec) is a convolutional autoencoder based on the DAC architecture, which holds SOTA. We found that training with EMA and adding a perceptual loss term with CLAP features improved performance. These codecs, being low compression, outperform Meta's EnCodec and DAC on general audio as validated from internal blind ELO games. We trained (relatively) very low compression codecs in the pursuit of solving a core issue regarding general music and audio generation, low acoustic quality, and audible artifacts, which hinder industry use for these models. Our hope is to encourage researchers to build hierarchical generative audio models that can efficiently use high sequence length representations without sacrificing semantic abilities. -
20
FlexLLMGen
Running large language models on a single GPU
FlexLLMGen is an open-source inference engine designed to run large language models efficiently on limited hardware resources such as a single GPU. The system focuses on high-throughput generation workloads where large batches of text must be processed quickly, such as large-scale data extraction or document analysis tasks. Instead of requiring expensive multi-GPU systems, the framework uses techniques such as memory offloading, compression, and optimized batching to run large models on commodity hardware. ... -
21
Vortex
The LLVM of columnar file formats
Vortex is a high-performance toolkit designed for working with compressed Apache Arrow arrays, providing functionality for in-memory, on-disk, and over-the-wire data handling. It aims to be an advanced successor to Apache Parquet, offering dramatically faster random access reads and scans, while maintaining similar compression ratios. Vortex's modular design allows for extensibility, enabling developers to implement custom encodings for efficient data management, particularly for large-scale columnar datasets. -
22
PaddleOCR
Awesome multilingual OCR toolkits based on PaddlePaddle
...Inspired by PaddlePaddle, PaddleOCR is an ultra lightweight OCR system, with multilingual recognition, digit recognition, vertical text recognition, as well as long text recognition. It features a PPOCR series of high-quality pre-trained models, which includes: ultra lightweight ppocr_mobile series models, general ppocr_server series models, and ultra lightweight compression ppocr_mobile_slim series models. PaddleOCR is easy to install and easy to use on Windows, Linux, MacOS and other systems. -
23
Ccache
A fast compiler cache
Ccache is a compiler cache. It speeds up recompilation by caching previous compilations and detecting when the same compilation is being done again. Ccache is free software, released under the GNU General Public License version 3 or later. Supports GCC, Clang, MSVC (Microsoft Visual C++) and other similar compilers. Works on Linux, macOS, other Unix-like operating systems and Windows. Understands C, C++, assembler, CUDA, Objective-C and Objective-C++. Supports secondary storage over HTTP... -
24
WavTokenizer
SOTA discrete acoustic codec models with 40/75 tokens per second
WavTokenizer is a state-of-the-art discrete acoustic codec designed specifically for audio language modeling, capable of compressing 24 kHz audio into just 40 or 75 tokens per second while preserving high perceptual quality. It is built to represent speech, music, and general audio with extremely low bitrate, making it ideal as a front-end for large audio language models like GPT-4o and similar architectures. The model uses a single-quantizer design together with temporal compression to achieve extreme compression without sacrificing reconstruction fidelity. ... -
25
libavif
Library for encoding and decoding .avif files
...The API is straightforward, allowing applications to integrate AVIF support natively — creating, decoding, compressing, or decompressing images programmatically. Recent developments have added experimental support for HDR gain maps, improved color-space handling (including wide color gamut and high bit depth), and better quality tuning modes for encoding, making libavif a cutting-edge tool for high-quality image compression.
