A CLI tool to aggregate your codebase into a single Markdown file for use with Claude Projects or custom ChatGPTs.
Start by running the CLI tool in your project directory:
npx ai-digest
This will generate a codebase.md
file with your codebase.
Once you've generated the Markdown file containing your codebase, you can use it with AI models like ChatGPT and Claude for code analysis and assistance.
- Aggregates all files in the specified directory and subdirectories
- Ignores common build artifacts and configuration files
- Outputs a single Markdown file containing the whole codebase
- Provides options for whitespace removal and custom ignore patterns
- View file size statistics with visual bar charts
- Watch mode for automatic rebuilding when files change
- Create a Custom GPT
- Upload the generated Markdown file to the GPT's knowledge base
- Create a new Project
- Add the Markdown file to the Project's knowledge
For best results, re-upload the Markdown file before starting a new chat session to ensure the AI has the most up-to-date version of your codebase.
-
-i, --input <directories...>
: Specify input directories (multiple allowed, default: current directory) -
-o, --output <file>
: Specify output file (default: codebase.md) -
--no-default-ignores
: Disable default ignore patterns -
--whitespace-removal
: Enable whitespace removal -
--show-output-files [sort]
: Display a list of files with size statistics and bar charts. Addsort
to sort by file size. -
--ignore-file <file>
: Specify a custom ignore file (default: .aidigestignore) -
--watch
: Enable watch mode to automatically rebuild when files change -
--help
: Show help
-
Basic usage:
npx ai-digest
-
Specify input and output:
npx ai-digest -i /path/to/your/project -o project_summary.md
-
Enable whitespace removal:
npx ai-digest --whitespace-removal
-
Show files included with size statistics:
npx ai-digest --show-output-files
-
Show files sorted by size (largest first):
npx ai-digest --show-output-files sort
-
Watch mode:
npx ai-digest --watch
-
Combine multiple options:
npx ai-digest -i /path/to/your/project -o project_summary.md --whitespace-removal --show-output-files sort --watch
ai-digest can also be used as a library in your Node.js projects:
npm install ai-digest
import aiDigest from 'ai-digest';
// Generate digest content as a string
const content = await aiDigest.generateDigest({
inputDir: './src',
outputFile: null, // Return as string instead of writing to file
silent: true // Suppress console output
});
// Or save directly to a file
await aiDigest.generateDigest({
inputDir: './src',
outputFile: 'codebase.md',
removeWhitespaceFlag: true,
showOutputFiles: 'sort'
});
// Get individual file objects for custom filtering
const { files } = await aiDigest.generateDigestFiles({
inputDir: './src',
silent: true
});
// Each file has: { fileName: string, content: string }
// Content format is identical to generateDigest() but per-file
console.log(files[0].fileName); // e.g., "index.ts"
console.log(files[0].content); // "# index.ts\n\n```ts\n// file content...\n```\n\n"
// Apply custom filtering after processing
const jsFiles = files.filter(file => file.fileName.endsWith('.js'));
const customDigest = jsFiles.map(file => file.content).join('');
// Get file statistics without content (useful for analysis)
const stats = await aiDigest.getFileStats({
inputDir: './src',
silent: true
});
// Returns files sorted by size (largest first) with total token counts
console.log(stats);
// {
// files: [
// { path: 'large-file.js', sizeInBytes: 15420 },
// { path: 'medium-file.ts', sizeInBytes: 8200 },
// { path: 'small-file.txt', sizeInBytes: 1024 }
// ],
// totalGptTokens: 5850,
// totalClaudeTokens: 6284
// }
Available functions:
-
generateDigest(options)
- Main function for generating digests -
generateDigestFiles(options)
- Generate digest and return array of individual file objects -
generateDigestContent(options)
- Lower-level function that returns content and stats -
writeDigestToFile(content, outputFile, stats)
- Write digest content to a file -
getFileStats(options)
- Get file statistics (path, size) sorted by size with total token counts, without content
ai-digest supports custom ignore patterns using a .aidigestignore
file in the root directory of your project. This file works similarly to .gitignore
, allowing you to specify files and directories that should be excluded from the aggregation.
Use the --show-output-files
flag to see which files are being included, making it easier to identify candidates for exclusion.
When using the --whitespace-removal
flag, ai-digest removes excess whitespace from files to reduce the token count when used with AI models. This feature is disabled for whitespace-dependent languages like Python and YAML.
Binary files and SVGs are included in the output with a note about their file type. This allows AI models to be aware of these files without including their full content.
When using the --watch
flag, ai-digest will continuously monitor your files for changes and automatically regenerate the output file whenever a relevant file is modified, added, or deleted. This is especially useful during development when you're making frequent changes to your codebase.
The watch mode:
- Respects all ignore patterns (both default and custom)
- Rebuilds only when non-ignored files change
- Includes a debounce mechanism to avoid multiple rebuilds when many files change at once
- Can be terminated with Ctrl+C
Run npm run start
to run the CLI tool on the local project. (Very meta!)
Run npm test
to run the tests.
To pass flags to the CLI, use the --
flag, like this: npm run start -- --whitespace-removal
.
npm publish
Contributions are welcome! Please feel free to submit a Pull Request.
This project is licensed under the MIT License.