Open Source Rust Large Language Models (LLM)

llmfit is a terminal-based utility that helps developers determine which large language models can realistically run on their local hardware by analyzing system resources and model requirements. The tool automatically detects CPU, RAM, GPU, and VRAM specifications, then ranks available models based on performance factors such as speed, quality, and memory fit. It provides both an interactive terminal user interface and a traditional CLI mode, enabling flexible workflows for different user preferences. llmfit also supports advanced configurations including multi-GPU setups, mixture-of-experts architectures, and dynamic quantization recommendations. By presenting clear performance estimates and compatibility guidance, the project reduces the trial-and-error typically involved in local LLM experimentation. Overall, llmfit serves as a practical decision assistant for developers who want to run language models efficiently on their own machines.

llm is an ecosystem of Rust libraries for working with large language models - it's built on top of the fast, efficient GGML library for machine learning. The primary entry point for developers is the llm crate, which wraps the llm-base and the supported model crates. Documentation for the released version is available on Docs.rs. For end-users, there is a CLI application, llm-cli, which provides a convenient interface for interacting with supported models. Text generation can be done as a one-off based on a prompt, or interactively, through REPL or chat modes. The CLI can also be used to serialize (print) decoded models, quantize GGML files, or compute the perplexity of a model. It can be downloaded from the latest GitHub release or by installing it from crates.io.

Cake is a compact, powerful toolkit that combines a flexible TCP/UDP proxy, port forwarding system, and connection manager designed for both development and penetration testing scenarios. It enables users to create complex networking flows where traffic can be proxied, relayed, and manipulated between endpoints — useful for debugging networked applications, inspecting protocols, or tunneling traffic through different hops. The tool is designed to work with multiple protocols and supports dynamic rule definitions so that incoming and outgoing connections can be routed, rewritten, or logged according to user-defined policies. Unlike many simple proxies, Cake can act as a full connection broker: it can bind to arbitrary interfaces, handle simultaneous upstream/downstream sessions, and apply traffic rules on the fly. This makes it suitable for troubleshooting tricky network behavior, simulating network conditions, or chaining services in a modular test environment.

This is HASH's public monorepo which contains our public code, docs, and other key resources. HASH is a platform for decision-making, which helps you integrate, understand and use data in a variety of different ways. HASH does this by combining various different powerful tools together into one simple interface. These range from data pipelines and a graph database, through to an all-in-one workspace, no-code tool builder, and agent-based simulation engine. These exist at varying stages of maturity, and while some are polished, not all are ready for real-world production use. You can read more about our big-picture vision at hash.dev

"Building Large Language Model Applications: Application Development and Architecture Design" is an open source e-book about the real-world application of LLM. It introduces the basics and applications of large language models, as well as how to build your own models. These include writing, developing, and managing prompts, exploring what the best large language models can bring, and pattern and architecture design for LLM application development.

A git prepare-commit-msg hook for authoring commit messages with GPT-3. With this tool, you can easily generate clear, comprehensive and descriptive commit messages letting you focus on writing code. To use gptcommit, simply run git commit as you normally would. The hook will automatically generate a commit message for you using a large language model like GPT. If you're not satisfied with the generated message, you can always edit it before committing. By default, gptcommit uses the GPT-3 model. Please ensure you have sufficient credits in your OpenAI account to use it. Commit messages are a key channel for developers to communicate their work with others, especially in code reviews. When making complex code changes, it can be tedious to thoroughly document the contents of each change. I often felt the impulse to just title my commit “fix bug” and move on. Surfacing these changes with gptcommit helps the author and reviewer by bringing attention to these additional changes.

Output from a language model using standard input as the prompt. Now supporting GPT3.5 chat completions! gptee was designed for use within shell scripts and other programs and also works in interactive shells. You can compose commands and execute them in a script. Proceed with caution before running arbitrary shell scripts. Using a chat completion model (like gpt-3.5-turbo), you can then inject a system message with -s or --system messages. For davinci and other non-chat models, the output is prefixed to the prompt. Compose shell commands like you would in a script. Try with a custom model. By default gptee uses gpt-3.5-turbo.

We offer a collection of Rust crates packed with features that make working with Large Language Models easy and seamless. With llm-chain, you can focus on building powerful AI applications. Create reusable and easily customizable prompt templates for consistent and structured interactions with LLMs. Build powerful chains of prompts that allow you to execute more complex tasks, step by step, leveraging the full potential of LLMs. Provides seamless integration with LLaMa models, enabling natural language understanding and generation tasks with Facebook's research models. Incorporates support for Stanford's Alpaca models, expanding the range of available language models for advanced AI applications. Enhance your AI agents' capabilities by giving them access to various tools, such as running Bash commands, executing Python scripts, or performing web searches.

pgvecto.rs is a Postgres extension that provides vector similarity search functions. It is written in Rust and based on pgrx. It is currently under heavy development, please take care when using it in production. pgvecto.rs is a Postgres extension, which means that you can use it directly within your existing database. This makes it easy to integrate into your existing workflows and applications. pgvecto.rs supports filtering. You can set conditions when searching or retrieving points. This is the missing feature of other postgres extensions.