Search Results for "remove duplicate lines"
Sort By:
Showing 112 open source projects for "remove duplicate lines"
View related business solutions-
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
RemoveDuplicateLinesFromTxt
Remove Duplicate lines From Files in TXT
This program remove duplicate lines in txt files. -
2
Text Line Duplicate Remover
Remove duplicate lines from your text
This standalone offline web browser tool helps you remove duplicate lines from your text, with additional text processing options. Simply open it in your browser by double-clicking the html file. It also includes the source code too. I made this when I was working with long lists of entries and needed something to automatically clean them up. As a bonus you can also change the Sentence Case of the text, make it lowercase, UPPERCASE or Sentence case. -
3
Video-subtitle-extractor
A GUI tool for extracting hard-coded subtitle (hardsub) from videos
Video hard subtitle extraction, generate srt file. There is no need to apply for a third-party API, and text recognition can be implemented locally. A deep learning-based video subtitle extraction framework, including subtitle region detection and subtitle content extraction. A GUI tool for extracting hard-coded subtitles (hardsub) from videos and generating srt files. Use local OCR recognition, no need to set up and call any API, and do not need to access online OCR services such as Baidu... -
4
FDUPES
FDUPES is a program for identifying or deleting duplicate files
FDUPES is a lightweight command-line utility that helps users find and optionally delete duplicate files within specified directories by comparing file contents, which can be extremely useful for cleaning up storage clutter or organizing large collections of files. It works by scanning directories and subdirectories, identifying sets of files with identical content through size and hash comparisons, and then listing them together so users can examine duplicates. -
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
5
RemoveDuplicate
A software that can helps you remove duplicate files
RemoveDuplicate is a Windows application designed to help users find and remove duplicate files within folders. It uses MD5 hashing to identify identical files and provides a user-friendly interface to manage and delete duplicates while keeping one copy of each file. -
6
Grabber
Very customizable imageboard/booru downloader with powerful filenaming
Very customizable imageboard/booru browser and mass downloader, with powerful file naming features. Thanks to its powerful naming features, you just have to set your filename and save the directory using all the tokens available, and the program will generate a filename using the image's information. With this, you can store and manage your pictures in advanced directory structures, and save images with custom filenames. Imageboard/booru downloader which can download thousands of images from... -
7
Discordeno
Discord API library for Node and Deno
...The functional API eliminates the challenges of extending built-in classes and inheritance while ensuring overall simple but performant code. Discordeno offers the option to have practically any component of a bot as a separate piece, including standalone REST, gateways, custom caches, and more. Remove any properties, if your bot doesn't need them. For instance, remove Channel.topic if your bot doesn't require it. You may save GBs of RAM in this way. A few lines of code are all that are needed to accomplish this for any property on any object. -
8
DupRem
Simple application to remove duplicate and empty lines on text files.
DupRem is a simple easy-to-use cross-platform application to remove duplicate and empty lines from any text file. It is also possible to keep or ignore case sensitive. Works also from command-line interface, e.g. "java -jar duprem.jar -r input_file.txt >output_file.txt", to create output file, or "java -jar duprem.jar -r input_file.txt >>output_file.txt", to create or append to output file. DupRem is portable, does not need installation and is developed in Java, so needs the Java Virtual Machine (version 6 or higher) that, in the case, is very easy to download and install. -
9
Swiss File Knife
One hundred command line tools in a small and portable binary.
Create zip files, extract zip files, replace text in files, search in files using expressions, stream text editor, instant command line ftp and http server, send folder via network, copy folder excluding sub folders and files, find duplicate files, run a command on all files of a folder, split and join large files, make md5 checksum lists of files, remove tab characters, convert CR/LF, list newest or biggest files of a folder, compare folders, treesize, show first or last lines of a file, find filenames fast using index files, rename many files using expressions, copy part of a file, change times of a file, set file time from filename, print colored text to terminal, convert csv to tab separated, download files from web, send http or udp requests, print tcp or udp traffic, create hexdump of files, join many text files into one, list nested .zip .tar .tar.gz .tar.bz2 archive contents. ... -
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
10
autopep8
A tool that automatically formats Python code to conform to the PEP 8
...Correct deprecated or non-idiomatic Python code (via lib2to3). Use this for making Python 2.7 code more compatible with Python 3. Put a blank line between a class docstring and its first method declaration. Remove blank lines between a function declaration and its docstring. -
11
Meta Package Manager
Wraps all package managers with a unifying CLI
Meta Package Manager wraps all package managers with a unifying CLI, and provides the MPM CLI, a wrapper around all package managers. MPM is like yt-dlp, but for package managers instead of videos. MPM solves XKCD #1654 - Universal Install Script. List installed packages. List duplicate installed packages. Search for packages. Install a package, remove a package, and list outdated packages. Sync local package infos. Upgrade all outdated packages. Backup list of installed packages to TOML file. Restore/install list of packages from TOML files. Pin-point commands to a subset of package managers (include/exclude selectors). ... -
12
Cirrus
The SCSS framework for the modern web
...Forget bundling tons of JavaScript with each component. All styles require no JS for interactions/functionality (see Modals, Dropdowns, etc.). Add additional components, remove utility classes, disable features, etc. Cirrus takes a generative approach to building your stylesheets. A large number of features with a minimal footprint. Fully responsive by design. Getting started with Cirrus is a few commands or lines of HTML away (depending on how you want to import it). The framework is designed to be pluggable into any new or existing projects. -
13
duf
Disk usage and free utility, a better 'df' alternative
...You can simply start duf without any command-line arguments. If you supply arguments, duf will only list specific devices & mount points. If you want to list everything (including pseudo, duplicate, inaccessible file systems), you can. You can show and hide specific tables. You can also show and hide specific filesystems. List inode information instead of block usage. If duf doesn't detect your terminal's colors correctly, you can set a theme. Users of oh-my-zsh should be aware that it already defines an alias called duf, which you will have to remove in order to use duf. -
14
ngx-quill
Angular components for the Quill Rich Text Editor
...In general, QuillJS recommends using a QuillJS instance to present your content. Just create a quill editor without a toolbar and in read-only mode. With some simple CSS lines you can remove the default border around the content. -
15
Amazon CodeGuru Profiler Python Agent
Amazon CodeGuru Profiler Python Agent
Amazon CodeGuru Profiler collects runtime performance data from your live applications and provides recommendations that can help you fine-tune your application performance. Using machine learning algorithms, CodeGuru Profiler can help you find your most expensive lines of code and suggest ways you can improve efficiency and remove CPU bottlenecks. CodeGuru Profiler provides different visualizations of profiling data to help you identify what code is running on the CPU, see how much time is consumed, and suggest ways to reduce CPU utilization. Use CodeGuru Profiler to help profile your applications in the cloud from a single, centralized dashboard. ... -
16
ParDRe
Parallel tool to remove duplicate DNA reads
ParDRe is a parallel tool to remove duplicate reads. Duplicate reads can be seen as identical or nearly identical sequences with some mismatches. This tool will let the users to avoid the analysis of not necessary reads, reducing the time of subsequent procedures with the dataset (e.g., assemblies, mappings, etc.). The tool is implemented with MPI in order to exploit the parallel capabilities of multicore clusters. -
17
SortMyShit
A tool designed to help you organize and manage your files
SortMyShit is an open-source Python project designed to help you organize and manage your files effortlessly. It provides customizable sorting rules to keep your directories clean and structured. -
18
Blaiz Tools
Tools for working with Gossamer codebase and app source code
Tools for packing files, converting images, manipulating text, checking code and working with the Gossamer codebase and app source code. -
19
Subtitle Editor
Subtitle Editor for SRT, SUB, SBV, TXT, ASS, SSA subs
Srt Editor is a small subtitle editing tool for Linux, OSX and Win64. You can edit the following file formats in various ways : SRT, SUB, TXT, SBV, ASS, SSA. Last version : 4.0.3 (MacOS & Linux) / 3.2.1 (Windows) -
20
logwitch
a simple log file scanner for linux written in shell and lua
logwitch checks single or rotated log files, either plain text or gzipped. It can work with log4j and gnu/linux system logs. You configure it to watch for lines in log files that interest you in by creating a simple text file in its config directory. It emails a daily report to you and timestamps logs so that you do not receive duplicate information. logwitch is GPL3 licensed -
21
UniFormat
This project process code format to a unified style.
This project is used to process code format to a unified style. First we can process C/C++ code, we will process other languages later. -
22
PyPingMon
A simple ping server monitor
PyPingMon is a server monitoring program written in Python. At the specified interval it uses a single ping command to check if a server is available. This is my first Python program. I chose this particular type of program because I wanted to write my own server monitor and this was a great way to learn how to code in Python. -
23
AvaSattva
Search replace files or pipe
See https://github.com/qualiu/msr/ Match/Search/Replace: msr.exe/msr-Win32.exe/msr.cygwin/msr.gcc**/msr-i386.gcc** Match/Search/Replace/Execute/* Files/Pipe Lines/Blocks. Filter/Load/Extract/Transform/Stats/* Files/Pipe Lines/Blocks. Not-IN-latter: nin.exe/nin-Win32.exe/nin.cygwin/nin.gcc**/nin-i386.gcc** Get Exclusive/Mutual Line-Set or Key-Set; Remove Line-Set or Key-Set matched in latter file/pipe; Get Unique/Mutual/Distribution/Stats/* Files/Pipe Line-Set or Key-Set. Match/Search/Replace files/pipe text with plain/Regex syntax. ... -
24
Image Comparator
Find and remove duplicate or similar images.
Image Comparator is a robust Windows desktop application designed to identify and manage duplicate or visually similar images using advanced Discrete Cosine Transform (DCT) algorithms. Unlike standard comparison tools, it effectively detects images that have been resized, compressed, or slightly modified. -
25
CVTRPGFREE
Convert fixed-format RPG code to free-form RPGILE code
This IBM i (aka iSeries, AS400) command provides for code modernisation by converting fixed-format RPG and RPGLE source code into fully free-form RPGLE source in a non-destructive manner. Converts: * H-Specs * F-Specs * D-Specs * C-Specs * P-Specs (Subprocedures) * Key Lists * Parameter Lists **PLEASE REPORT ANY ISSUES YOU FIND** NOTE: Requires V7R2M0 or higher. Some Background: This started out as a 'quick and dirty' program for my own needs, but it got a little out of...
