Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Completed Suggestions

Items from suggestions.md that have been implemented.

Completed Features

  • Remote caching — See Remote Caching. Share build artifacts across machines via S3, HTTP, or filesystem.
  • Lua plugin system — built, then removed on 2026-10-17: no repository ever declared a [processor.lua.*] stanza, and the host could not take part in cache versioning or field validation. Custom commands go through the config-driven processors (script, generic) instead.
  • Tool version locking — rsconstruct tool lock locks and verifies external tool versions. Tool versions are included in cache keys.
  • JSON output mode — --json flag for machine-readable JSON Lines output (build_start, product_start, product_complete, build_summary events).
  • Native C/C++ include scanner — Default include_scanner = "native" uses regex-based scanning. Falls back to include_scanner = "compiler" (gcc -MM).
  • --processor flag — rsconstruct build -p processor.generator.tera,processor.checker.ruff and rsconstruct watch -p processor.generator.tera filter which processors run.
  • Colored diff on config changes — When processor config changes trigger rebuilds, rsconstruct shows what changed with colored diff output.
  • Batch processing — ruff, pylint, shellcheck, zspell, mypy, and rumdl all support batch execution via execute_batch().
  • Progress bar — Uses indicatif crate. Progress bar sized to actual work (excludes instant skips), hidden in verbose/JSON mode.
  • Emit ProductStart JSON events — Emitted before each product starts executing, pairs with ProductComplete for per-product timing.
  • mypy processor — Python type checking with mypy. Batch-capable. Auto-detects mypy.ini as extra input.
  • Explain commands — --explain flag shows skip/restore/rebuild reasons for each product during build.
  • -p pulls in the producers of its inputs — -p B runs B and the processors that generate B’s inputs, transitively, as --target does for products; every processor is discovered and the selection is made on the full graph (select_processors). -x excludes a processor and everything downstream of it, and an excluded processor is never pulled back in.
  • Unchanged-output pruning — When a product rebuilds to identical bytes, the products reading it are skipped with their outputs untouched: no rerun, no restore. Each product is decided at dispatch from the inputs it will read (R1), and its stale outputs are removed only right before it actually restores or rebuilds — no longer for every predicted change before the run. Products predicted to change that never run (a failed dependency, an error, Ctrl+C) have their stale outputs removed at the end of the run, so a failed build still leaves none behind. Processors whose one tool run writes many products’ outputs (mass generators) replace them themselves (Processor::replaces_own_outputs).
  • Ready-queue scheduler — Products run the moment their own dependencies finish, from a pool of exactly -j workers, instead of level by level. Batch chunks run in parallel and count against -j; max_jobs is enforced by the queue instead of semaphores. See R6/R7 in Architecture Observations.

Completed Code Consolidation

  • Collapsed checker_config! macro variants — Merged @basic, @with_auto_inputs, and @with_linter into two internal variants (@no_linter and @with_linter).
  • Added batch field to all manually-defined processor configs — All processor configs now support batch = false to disable batching per-project.
  • Replaced trivial checker files with simple_checker! macro — 25 trivial checkers reduced from ~35 lines each to 3-5 lines (~800 lines eliminated).
  • Unified lint_files/check_files naming — All checkers now use check_files consistently.
  • Moved should_process guard into macro — Added guard: scan_root built-in to impl_checker!, removed boilerplate should_process() from 7 processors.
  • Simplified KnownFields — Scan config fields auto-appended by validation layer via SCAN_CONFIG_FIELDS constant; KnownFields impls only list their own fields.
  • Extracted WordManager for spellcheck/aspell — Shared word-file management (loading, collecting, flushing, execute/batch patterns) in word_manager.rs.

Completed New Processors

  • mypy — Python type checking using mypy. Batch-capable. Config: checker, args, dep_inputs, scan.
  • yamllint — Lint YAML files using yamllint. src/processor/checker/yamllint.rs.
  • jsonlint — Validate JSON files for syntax errors. src/processor/checker/jsonlint.rs.
  • taplo (toml-lint) — Validate TOML files using taplo. src/processor/checker/taplo.rs.
  • markdownlint — Lint Markdown files for structural issues. Uses mdl or markdownlint-cli.
  • pandoc — Convert Markdown to other formats (PDF, HTML, EPUB). Generator processor.
  • jinja2 — Render Jinja2 templates (.j2) via Python jinja2 library. src/processor/generator/jinja2.rs.
  • black — Python formatting verification using black --check. src/processor/checker/black.rs.
  • rust_single_file — Compile single-file Rust programs to executables. src/processor/generator/rust_single_file.rs.
  • sass — Compile SCSS/SASS files to CSS. src/processor/generator/sass.rs.
  • protobuf — Compile .proto files to generated code using protoc. src/processor/generator/protobuf.rs.
  • pytest — Run Python test files with pytest. src/processor/checker/pytest.rs.
  • doctest — Run Python doctests via python3 -m doctest. src/processor/checker/doctest.rs.

Completed Test Coverage

  • Ruff/pylint processor tests — tests/processor/ruff.rs and tests/processor/pylint.rs with integration tests.
  • Make processor tests — tests/processor/make.rs with Makefile discovery and execution tests.
  • All generator processor tests — Integration tests for all 14 previously untested generators: a2x, drawio, gem, libreoffice, markdown, marp, mermaid, npm, pandoc, pdflatex, pdfunite, pip, sphinx.
  • All checker processor tests — Integration tests for all 5 previously untested checkers: ascii, aspell, markdownlint, mdbook, mdl.

Completed Caching & Performance

  • Lazy file hashing (mtime-based) — mtime_check config (default true), fast_checksum() with MTIME_TABLE. Stores (path, mtime, checksum) tuples. Disable with --no-mtime.
  • Compressed cache objects — Optional zstd compression for .rsconstruct/objects/. Config: compression = true in [cache]. Incompatible with hardlink restore (must use restore_method = "copy"). Checksums computed on original content for stable cache keys.

Completed Developer Experience

  • --quiet flag — -q/--quiet suppresses all output except errors. Useful for CI scripts that only care about exit code.
  • Flaky product detection / retry — --retry=N retries failed products up to N times. Reports FLAKY (passed on retry) vs FAILED status in build summary.
  • Actionable error messages — rsconstruct tool check shows install hints for missing tools (e.g., “install with: pip install ruff”).
  • Build profiling / tracing — --trace=file.json generates Chrome trace format output viewable in chrome://tracing or Perfetto UI.
  • rsconstruct build <target> — Build specific targets by name or pattern via --target glob patterns and -d/--dir flags.
  • rsconstruct why <file> / Explain rebuilds — --explain flag shows why each product is skipped, restored, or rebuilt.
  • rsconstruct doctor — Diagnose build environment: checks config, tools, and versions. Full implementation in src/builder/doctor.rs.
  • rsconstruct sloc — Source lines of code statistics with COCOMO effort/cost estimation. src/builder/sloc.rs.

Completed Quick Wins

  • Batch processing for more processors — All checker processors that support multiple file arguments now use batching.
  • Progress bar for long builds — Implemented with indicatif, shows [elapsed] [bar] pos/len message.
  • --processor flag for build and watch — Filter processors with -p flag.
  • Emit ProductStart JSON events — Wired up and emitted before execution.
  • Colored diff on config changes — Shows colored JSON diff when processor config changes.

Completed Features (v0.3.7)

  • RSCONSTRUCT_THREADS env var — Set parallelism via environment variable instead of -j. Priority: CLI -j > RSCONSTRUCT_THREADS > config parallel.
  • Global output_dir in [build] — Global output directory prefix (default: "out"). Processor defaults like out/processor.generator.marp are remapped when the global is changed (e.g., output_dir = "build" makes marp output to build/marp). Individual processors can still override their output_dir explicitly.
  • Named processor instance output directories — When multiple instances of the same processor are declared (e.g., [processor.generator.marp.slides] and [processor.generator.marp.docs]), each instance defaults to out/{instance_name} (e.g., out/marp.slides, out/marp.docs) instead of sharing the same output directory.
  • Named processor instance names in error reporting — When multiple instances of the same processor exist, error messages, build progress, and statistics use the full instance name (e.g., [pylint.core], [pylint.tests]). Single instances continue to use just the processor type name.
  • processor config without config file — rsconstruct processor config <name> now works without an rsconstruct.toml, showing the default configuration (same as defconfig).
  • tag collect command — rsconstruct tag collect scans the tags database for tags that are not in the tag collection (tags_dir) and adds them to the appropriate .txt files. Key:value tags go to {key}.txt, bare tags go to tags.txt.
  • rsconstruct status shows 0-file processors — Processors declared in the config that match no files are now shown in status output and the --breakdown summary, making it easy to spot misconfigured or unnecessary processors.
  • smart remove-no-file-processors — New command rsconstruct smart remove-no-file-processors removes [processor.*] sections from rsconstruct.toml for processors that don’t match any files. Handles both single and named instances.
  • cc_single_file output_dir from config — The cc_single_file processor now reads its output directory from the config output_dir field instead of hardcoding out/processor.generator.cc_single_file. This fixes named instances (e.g., cc_single_file.gcc and cc_single_file.clang) which previously collided on the same output directory.
  • clean unknown respects .gitignore — rsconstruct clean unknown now skips gitignored files. Previously it disabled .gitignore handling, causing intentionally ignored files (IDE configs, virtualenvs, *.pyc, etc.) to be flagged as unknown. RSConstruct outputs are still correctly identified via the build graph, so nothing is missed. Use --no-gitignore to include gitignored files.
  • Cross-processor dependencies (fixed-point discovery) — Generator outputs are now visible to downstream processors on the first build. Discovery runs in a fixed-point loop: after each pass, declared outputs are injected as virtual files into the FileIndex, and discovery re-runs until no new products are found. This means a generator that creates .md files can feed pandoc/tags/spell-checkers in a single build, without needing a second build.

Completed Architecture Refactors

  • Config provenance tracking — Every config field now carries FieldProvenance (UserToml with line number, ProcessorDefault, ScanDefault, OutputDirDefault, SerdeDefault). rsconstruct config show annotates every field with its source. Uses toml_edit::Document for span capture.
  • BuildContext replacing process globals — All mutable process globals moved into BuildContext: the three processor globals (INTERRUPTED, RUNTIME, INTERRUPT_SENDER) and the three checksum globals (CACHE, MTIME_DB, MTIME_ENABLED). Threaded through the Processor trait, executor, analyzers, remote cache, checksum functions, and deps cache. Signal handler uses Arc<BuildContext>.
  • BuildPolicy trait — Extracted from the executor. classify_products delegates per-product skip/restore/rebuild decisions to a &dyn BuildPolicy. IncrementalPolicy implements the current logic. Future policies (dry-run, always-rebuild, time-windowed) are a single trait impl.
  • ObjectStore decomposition — mod.rs split from 664 → 223 lines into focused submodules: blobs.rs (content-addressed storage), descriptors.rs (cache descriptor CRUD), restore.rs (restore/needs_rebuild/can_restore/explain).

Completed Features (latest)

  • rsconstruct status --json — JSON output with per-processor counts (up_to_date, restorable, stale, new, total, native) and totals. Activated by --json flag.
  • Selective processor cleaning — rsconstruct clean outputs -p processor.checker.ruff,processor.checker.pylint cleans only those processors’ outputs. Without -p, cleans everything.
  • Prettier processor — Checker using prettier --check. Batch-capable. Scans .js/.jsx/.ts/.tsx/.mjs/.cjs/.css/.scss/.less/.html/.json/.md/.yaml/.yml. src/processor/checker/prettier.rs.
  • Bare clean requires subcommand — rsconstruct clean now errors with usage hint instead of silently defaulting to clean outputs.
  • Nondeterministic test race fix — Fixed TOCTOU race in store_descriptor where parallel writers could get Permission denied. Now retries after forcing writable on first failure.
  • Suppress status line for non-build commands — The Exited with SUCCESS/ERROR footer only shows for build, watch, and clean.
  • Configurable graph validation — Four checks run after resolve_dependencies(): (1) reject empty inputs (default on), (2) validate dep references (default on), (3) detect duplicate inputs within same processor (default off), (4) early cycle detection (default off). Config: [graph] section fields validate_empty_inputs, validate_dep_references, validate_duplicate_inputs, validate_early_cycles.
  • Checksum globals moved to BuildContext — CACHE, MTIME_DB, MTIME_ENABLED moved from src/checksum.rs statics into BuildContext. combined_input_checksum, checksum_fast, file_checksum all take &BuildContext. Completes the isolated-build-context story.
  • rsconstruct fix command — Runs fixers (auto-format, auto-fix) on source files. Checkers declare fix capability via fix_subcommand/fix_prepend_args on SimpleCheckerParams. processor list shows a Fix column. Supports -p filtering, batch execution, and --json. Fix-capable processors: ruff, black, prettier, eslint, stylelint, standard, taplo, rumdl, markdownlint. (Partly reverted 2026-08-04: fix capability was unused, so all nine can_fix flags were set to false; today only script instances with a fix_command are fix-capable, and processors are named as positional arguments to rsconstruct fix run rather than with -p. Removed entirely on 2026-10-17: the one fleet use was a script instance, and nothing else ever declared a fixer.)
  • processor search — rsconstruct processor search <query> searches by name, description, and keywords. All 91 processors have keywords covering language, tool category, file extensions, and ecosystem terms. Supports --json output.