Download Latest Version
v0.5.1 source code.tar.gz (583.7 kB)
Get an email when there's a new version of Chonkie
| Name | Modified | Size | Downloads / Week |
|---|---|---|---|
| Parent folder | |||
| README.md | 2024-12-29 | 1.4 kB | |
| v0.4.0 source code.tar.gz | 2024-12-29 | 421.4 kB | |
| v0.4.0 source code.zip | 2024-12-29 | 452.8 kB | |
| Totals: 3 Items | 875.6 kB | 0 | |
Highlights
Added the new RecursiveChunker that uses complex recursive rules to create structurally meaningful chunks, maintaining natural separations as much as possible. Try it out~
:::python
from chonkie import RecursiveChunker
chunker = RecursiveChunker(chunk_size=512)
chunks = chunker("Woah! Chonkie has it's own recursive chunker now~ so cooool!")
What's Changed
- Add initial support for Recursive Chunking (
RecursiveChunker) by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/107 - [FEAT] Add support for RecursiveChunking + minor fixes by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/108
- [fix] Correct the start and end indices for TokenChunker in Batch mode (#84) by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/109
- [fix] Correct the start and end indices for TokenChunker in Batch mode (#84) by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/110
- [fix] [#106]: Missing last sentence in the SemanticChunker by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/112
- [fix] Add fix for [#106]: Reconstruction tests for SemanticChunker failing, missing last sentence by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/113
- [chore] Bump version to "v0.4.0" + minor change by @bhavnicksm in https://github.com/chonkie-ai/chonkie/pull/114
Full Changelog: https://github.com/chonkie-ai/chonkie/compare/v0.3.0...v0.4.0
