This document explains the architecture and design decisions of obuild, a parallel, incremental, and declarative build system for OCaml.
- Genesis and Philosophy
- Architectural Layers
- Core Data Flow
- The Two-DAG Architecture
- Dependency Resolution System
- Build Execution Pipeline
- Type System Design
- Ctypes.cstubs Integration
- Custom Code Generators
- PPX and Preprocessor Resolution
- Configuration Files
- CLI Reference
- Design Decisions and Trade-offs
Obuild started on a bank holiday after xmas, as an experiment to make the simplest OCaml build system. The main goals are to:
- provide a good user experience.
- provide a building black box, no mocking around generic rules.
- provide features in the highest level possible.
- the cleanest build possible, with by-products hidden from the user.
- provide good defaults, and standardize names as much as possible.
- expose everything that the user of the build system needs in one place.
- be simple to build to prevent any bootstrapping problem.
One of the main influences was Haskell Cabal, which provides to all Haskellers a simple way to provide a build system to a project with a single file. This applies well for the myriad of OCaml options too.
Obuild is buildable with just the compiler and the compiler standard library. This make bootstrapping very easy: all you need is the OCaml compiler installed.
This creates some pain for developers of obuild, as lots of basic functions available in others libraries need to written again as part of obuild. As the initial development was done really quickly, some functions are not as performant (CPU or memory-wise) as they could be. This can be fixed as problem becomes apparent in scaling.
Each project is described really simply in a one place, in a user friendly format. A central .obuild file is used, and provides high level description of your project. Along with some meta data (name, authors, description, etc), it defines the library, and/or executable that the project wants to have, from which inputs (source files, modules).
All dependencies are fully autogenerated internally and used to recompile only the necessary bits.
Along with library and executable, test and benchmark can be defined, so as to provide an easy way to test or bench some part of your project. It also provides a standard on how to build and execute tests and benchmarks.
┌─────────────────────────────────────────────────────────────────┐
│ OBUILD DESIGN GOALS │
├─────────────────────────────────────────────────────────────────┤
│ Declarative │ Users describe targets, not build steps │
│ Parallel │ Independent tasks run concurrently │
│ Incremental │ Only rebuild what changed │
│ Self-contained │ No external build tool dependencies │
│ Discoverable │ Auto-detect modules, dependencies │
└─────────────────────────────────────────────────────────────────┘
Obuild is organized in four distinct layers, each with clear responsibilities:
┌─────────────────────────────────────────────────────────────────┐
│ LAYER 4: CLI │
│ src/main.ml │
│ Command dispatcher, user interface │
├─────────────────────────────────────────────────────────────────┤
│ LAYER 3: COMMANDS │
│ lib/help.ml, init.ml, install.ml, etc. │
│ High-level operations (init, install, doc) │
├─────────────────────────────────────────────────────────────────┤
│ LAYER 2: CORE │
│ lib/core/*.ml │
│ Parsing, analysis, dependency resolution, build execution │
├─────────────────────────────────────────────────────────────────┤
│ LAYER 1: BASE │
│ lib/base/*.ml │
│ Filepath, Filesystem, Fugue (utilities), CLI │
└─────────────────────────────────────────────────────────────────┘
The foundation layer has no dependencies on OCaml build tools. This is critical for bootstrapping - obuild must compile itself before it can use itself.
| Module | Purpose |
|---|---|
compat.ml |
OCaml version compatibility shims |
filepath.ml |
Type-safe path abstractions |
filesystem.ml |
File I/O operations |
fugue.ml |
Functional utilities (string, list, option) |
cli.ml |
Command-line parsing framework |
This layer contains ~60 modules implementing the build system logic:
lib/core/
├── Parsing ──────────── obuild_lexer.ml, obuild_parser.ml,
│ obuild_ast.ml, obuild_validate.ml,
│ project.ml, project_read.ml, meta.ml
│
├── Types ────────────── types.ml, target.ml, libname.ml,
│ modname.ml, hier.ml, filetype.ml
│
├── Analysis ─────────── analyze.ml, prepare.ml, prepare_types.ml,
│ dependencies.ml, metacache.ml
│
├── Execution ────────── build.ml, scheduler.ml, taskdep.ml,
│ process.ml, prog.ml, buildprogs.ml
│
├── Configuration ────── configure.ml, gconf.ml, findlibConf.ml
│
└── Utilities ────────── dag.ml, dagutils.ml, utils.ml, helper.ml
High-level operations that users invoke:
| Module | Command | Purpose |
|---|---|---|
help.ml |
obuild help |
Documentation system |
init.ml |
obuild init |
Create new projects |
install.ml |
obuild install |
Install artifacts |
doc.ml |
obuild doc |
Generate documentation |
sdist.ml |
obuild sdist |
Create source distributions |
The CLI dispatcher that:
- Parses command-line arguments
- Dispatches to appropriate command handler
- Manages global configuration
- Handles top-level error reporting
The build process follows a clear pipeline from configuration to compiled artifacts:
┌──────────────────────────────────────────────────────────────────────────┐
│ BUILD PIPELINE │
└──────────────────────────────────────────────────────────────────────────┘
.obuild file META files Source files
│ │ │
▼ ▼ ▼
┌─────────────┐ ┌─────────────────┐ ┌───────────────┐
│ PARSING │ │ DEPENDENCY │ │ MODULE │
│ │ │ RESOLUTION │ │ ANALYSIS │
│ Lexer │ │ │ │ │
│ Parser │ │ FindlibConf │ │ ocamldep │
│ Validator │ │ Meta parser │ │ Hier mapping │
│ │ │ Metacache │ │ │
└──────┬──────┘ └────────┬────────┘ └───────┬───────┘
│ │ │
▼ ▼ ▼
┌──────────────────────────────────────────────────────────────────────────┐
│ ANALYSIS │
│ (analyze.ml) │
│ │
│ Combines all inputs into project_config with resolved dependencies │
└──────────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────┐
│ PREPARATION │
│ (prepare.ml) │
│ │
│ For each target: scan modules, build compilation DAG, compute paths │
└──────────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────┐
│ EXECUTION │
│ (build.ml) │
│ │
│ Scheduler runs compile_steps in parallel, respecting dependencies │
└──────────────────────────────────────────────────────────────────────────┘
│
▼
Compiled Artifacts
(dist/build/<target>/...)
User main.ml Project_read Analyze Prepare Build Scheduler
│ │ │ │ │ │ │
│──build───────>│ │ │ │ │ │
│ │──read()─────>│ │ │ │ │
│ │ │──parse──────│ │ │ │
│ │ │──validate───│ │ │ │
│ │<──Project.t──│ │ │ │ │
│ │ │ │ │ │ │
│ │──prepare()───────────────>│ │ │ │
│ │ │ │──resolve───│ │ │
│ │ │ │ META │ │ │
│ │ │ │ files │ │ │
│ │<──project_config───────────│ │ │ │
│ │ │ │ │ │ │
│ │──build_dag()────────────────────────────────────────>│ │
│ │ │ │ │ │ │
│ │ │ │──prepare_target()──────>│ │
│ │ │ │<──compilation_state─────│ │
│ │ │ │ │ │ │
│ │ │ │ │──schedule()────────────>│
│ │ │ │ │ │ ┌────────┤
│ │ │ │ │ │ │parallel│
│ │ │ │ │ │ │compile │
│ │ │ │ │ │ │steps │
│ │ │ │ │ │ └────────┤
│ │ │ │ │<───────────────done──────│
│<──success─────│ │ │ │ │ │
A distinctive feature of obuild is its use of two separate DAGs (Directed Acyclic Graphs) for dependency tracking. Understanding why requires understanding two different questions:
The Files DAG answers this question. It tracks file-level dependencies:
Files DAG Example (for module Bar that uses Foo):
bar.cmo
/ | \
/ | \
bar.ml bar.cmi foo.cmi
|
bar.mli
- Nodes: Individual files (.ml, .mli, .cmi, .cmo, .cmx, .c, .h, .o)
- Edges: "file A depends on file B" (based on content)
- Purpose: Check modification times to determine what needs rebuilding
When foo.ml changes but foo.mli doesn't:
bar.cmodoesn't need rebuilding (it only depends onfoo.cmi)- Only
foo.cmoneeds recompilation
The Steps DAG answers this question. It tracks task execution order:
Steps DAG Example:
LinkTarget(mylib)
/ | \
/ | \
CompileModule CompileModule CompileC
(Bar) (Foo) (stubs.c)
| |
CompileInterface CompileInterface
(Bar) (Foo)
- Nodes: Compilation tasks (CompileModule, CompileInterface, CompileC, LinkTarget)
- Edges: "task A must complete before task B"
- Purpose: Enable parallel scheduling while respecting dependencies
| Aspect | Files DAG | Steps DAG |
|---|---|---|
| Purpose | Incremental builds | Parallel execution |
| Answers | "What changed?" | "What order?" |
| Granularity | Individual files | Compilation tasks |
| Usage | mtime comparison | Topological sort |
Separating these concerns allows:
- Fine-grained incrementality: Only recompile files whose dependencies changed
- Maximum parallelism: Independent tasks run concurrently
- Correct ordering: Dependent tasks wait for prerequisites
Obuild resolves dependencies through OCamlfind's META file system. This is one of the more complex subsystems.
ocamlfind is the current de-facto standard for installed package querying. ocamlfind is usually injected on the command line to ocamlopt, ocamldep, ocamlc with special flags (-syntax, -package), that ocamlfind will re-write to call the program with something that the program can understand. All the information for this transformation is stored in META files.
Unfortunately this design prevents META caching, and each time ocamlc/ocamlopt is used it will reparse the META files. This also causes problems if ocamlfind does not exist when used as a program, or if the library is not installed when used as a library.
Because of those 2 reasons, obuild has its own implementation of META parsing with caching support.
Search Order for library "base":
1. Check each path in OCAMLPATH:
├── /home/user/.opam/default/lib/base/META ← Found!
├── /usr/lib/ocaml/base/META
└── ...
2. Alternative: META.<name> format:
└── /usr/lib/ocaml/META.base
META file (text)
│
▼
┌─────────────┐
│ LEXER │
│ (meta.ml) │
└──────┬──────┘
│ tokens
▼
┌─────────────┐
│ PARSER │
│ (meta.ml) │
└──────┬──────┘
│
▼
┌─────────────┐
│ Pkg.t │
│ (cached) │
└─────────────┘
A critical design decision is how META files are cached:
┌─────────────────────────────────────────────────────────────────┐
│ METACACHE │
│ │
│ Hashtbl: main_name → (filepath, root_pkg) │
│ │
│ "base" → ("/path/to/base/META", <Pkg.t for base>) │
│ "str" → ("/path/to/str/META", <Pkg.t for str>) │
│ │
└─────────────────────────────────────────────────────────────────┘
│
│ get_from_cache("base.shadow_stdlib")
▼
Returns: (filepath, ROOT package for "base")
│
│ Caller must resolve subpackages:
│ Meta.Pkg.find(["shadow_stdlib"], root)
▼
Resolved subpackage Pkg.t
Key design: Metacache always returns the root package. Callers are responsible for navigating to subpackages. This prevents:
- Double resolution (cache pre-resolving, then caller resolving again)
- Incorrect path generation for subpackages
META files specify directories using special syntax:
Directory Field Resolves To
─────────────────────────────────────────────────
"" or "." basePath (where META lives)
"^" parent directory
"^subdir" parent/subdir
"+stdlib" $OCAMLLIB/stdlib
"/absolute/path" /absolute/path
"relative/path" basePath/relative/path
Analyze.prepare() builds multiple graphs:
┌─────────────────────────────────────────────────────────────────┐
│ project_pkgdeps_dag │
│ (Complete dependency picture) │
│ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │Exe:main │────────>│Lib:mylib│────────>│Dep:base │ │
│ └─────────┘ └────┬────┘ └────┬────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────┐ ┌─────────┐ │
│ │Dep:unix │ │Dep:sexp │ │
│ └─────────┘ └─────────┘ │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ project_targets_dag │
│ (Internal targets only) │
│ │
│ ┌─────────┐ ┌─────────┐ │
│ │Exe:main │──────────────────>│Lib:mylib│ │
│ └─────────┘ └─────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ project_dep_data │
│ (Dependency classification) │
│ │
│ mylib → Internal (defined in project) │
│ base → System (from OCamlfind) │
│ unix → System (from OCamlfind) │
│ │
└─────────────────────────────────────────────────────────────────┘
The scheduler manages parallel job execution with dependency tracking:
┌─────────────────────────────────────────────────────────────────┐
│ SCHEDULER │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Ready │ │ Running │ │ Completed │ │
│ │ Queue │───>│ Jobs │───>│ Tasks │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ ▲ │ │
│ │ │ │
│ └──────────────────────────────────────┘ │
│ When dependencies are satisfied │
│ │
└─────────────────────────────────────────────────────────────────┘
Each step represents an atomic compilation task:
type compile_step =
| CompileModule of Hier.t (* .ml → .cmo/.cmx *)
| CompileInterface of Hier.t (* .mli → .cmi *)
| CompileDirectory of Hier.t (* Pack directory modules *)
| CompileC of filename (* .c → .o *)
| GenerateCstubsTypes of Libname.t (* ctypes type discovery *)
| GenerateCstubsFunctions of Libname.t (* ctypes stub generation *)
| CompileCstubsC of Libname.t (* Compile generated C *)
| LinkTarget of target (* Link library/executable *)
| CheckTarget of target (* Verify outputs exist *)For library "mylib" with modules Foo, Bar (Bar depends on Foo):
Time ──────────────────────────────────────────────────────────>
Thread 1: ┌──────────────────┐
│CompileInterface │
│ (Foo) │
└────────┬─────────┘
│
┌────────▼─────────┐
│ CompileModule │
│ (Foo) │
└────────┬─────────┘
│
▼ (Bar can now start)
┌──────────────────┐
│ CompileModule │
│ (Bar) │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ LinkTarget │
│ (mylib) │
└──────────────────┘
Thread 2: ┌──────────────────┐
│CompileInterface │ (parallel with Foo interface)
│ (Bar) │
└──────────────────┘
│
│ (waits for CompileModule(Foo))
▼
┌──────────────────┐
│ CompileC │ (parallel with Bar module)
│ (stubs.c) │
└──────────────────┘
Obuild uses domain-specific types to prevent common errors:
(* lib/base/filepath.ml *)
type filepath (* Directory or file path *)
type filename (* Just a filename, no directory *)
(* These are abstract types - you cannot mix them up *)
val (</>) : filepath -> filename -> filepath (* Combine path + name *)
val (<//>) : filepath -> filepath -> filepath (* Combine paths *)┌─────────────────────────────────────────────────────────────────┐
│ NAME TYPES │
│ │
│ Libname.t │
│ ├── main_name: "base" │
│ └── subnames: ["shadow_stdlib"] │
│ → "base.shadow_stdlib" │
│ │
│ Modname.t │
│ └── Validated OCaml module name │
│ → "Base" (capitalized, valid chars) │
│ │
│ Hier.t │
│ └── Module hierarchy: [Base; Utils; String_extra] │
│ → "Base.Utils.String_extra" (module path) │
│ → "base/utils/string_extra" (file path) │
│ │
│ Target.Name.t │
│ ├── Lib of Libname.t │
│ ├── Exe of string │
│ ├── Test of string │
│ ├── Bench of string │
│ └── Example of string │
│ │
└─────────────────────────────────────────────────────────────────┘
(* WRONG: Easy to confuse in stringly-typed code *)
let path = dir ^ "/" ^ name ^ ".ml" (* What if name has a slash? *)
(* RIGHT: Types prevent mistakes *)
let path = dir </> (fn (name ^ ".ml")) (* Type-checked! *)Obuild supports ctypes.cstubs for generating C bindings declaratively.
library mylib
modules: Bindings, C, Types_generated
build-deps: ctypes, ctypes.stubs
cstubs
external-library-name: mylib_stubs
type-description: Bindings.Types -> Types_gen
function-description: Bindings.Functions -> Funcs_gen
generated-types: Types_generated
generated-entry-point: C
headers: string.h
.obuild configuration
│
│ cstubs block parsed
▼
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 1: Type Discovery │
│ │
│ Generate discover.ml ───> Compile ───> Run ───> discover.c │
│ │ │
│ Compile discover.c ───> Run ───> types_generated.ml │
│ │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 2: Compile Bindings │
│ │
│ bindings.ml (user's functor definitions) │
│ │ │
│ ▼ │
│ Compile bindings.cmo │
│ │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 3: Stub Generation │
│ │
│ Generate stubgen.ml ───> Compile with bindings.cmo ───> Run │
│ │ │
│ ├───> mylib_stubs_generated.ml (FOREIGN implementation) │
│ ├───> c.ml (entry point) │
│ └───> mylib_stubs.c (C stubs) │
│ │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 4: Compile Everything │
│ │
│ Compile generated .ml files │
│ Compile mylib_stubs.c ───> mylib_stubs.o │
│ Archive ───> libmylib_stubs.a │
│ Link everything into final library │
│ │
└─────────────────────────────────────────────────────────────────┘
LinkTarget(mylib)
/ | \
/ | \
CompileModule CompileCstubsC CompileModule
(C - entry) (stubs.c) (Mylib_stubs_generated)
| | |
└───────────────┼───────────────────┘
│
GenerateCstubsFunctions
│
┌──────────┴──────────┐
│ │
CompileModule(Bindings) GenerateCstubsTypes
│
CompileModule(Types_generated)
Obuild supports custom code generators defined in the .obuild file. This allows any code generation tool (menhir, ocamllex, atdgen, protobuf, etc.) to be integrated without hardcoding support in obuild itself.
Generators are registered during project parsing and invoked during module resolution. When obuild looks for a module and cannot find a .ml file, it checks for source files matching registered generator suffixes.
Module resolution for "Parser":
1. Look for parser.ml in src-dir → Not found
2. Check registered generators:
├── suffix "mly" → look for parser.mly → Found!
└── suffix "mll" → look for parser.mll → Not found
3. Run menhir generator on parser.mly
4. Produces parser.ml (and parser.mli)
5. Continue compilation with generated files
Generators are defined in lib/core/generators.ml with two key types:
(* Built-in generator *)
type t = {
suffix : string; (* File extension to match *)
modname : Modname.t -> Modname.t; (* Module name transformation *)
commands : filepath -> filepath -> string -> string list list;
generated_files : filename -> string -> filename;
}
(* Custom generator from .obuild file *)
type custom = {
custom_name : string; (* Unique identifier *)
custom_suffix : string option; (* Auto-detection suffix *)
custom_command : string; (* Command template *)
custom_outputs : string list; (* Output file patterns *)
custom_module_name : string option; (* Optional module name pattern *)
}Generator commands support template variables:
| Variable | Description |
|---|---|
${src} |
Full path to source file |
${dest} |
Destination path without extension |
${base} |
Base filename without extension |
${srcdir} |
Source directory |
${destdir} |
Destination directory |
${sources} |
Space-separated list of all inputs (multi-input only) |
.obuild file Module resolution Build execution
│ │ │
│ generator menhir │ Parser not found │
│ suffix: mly │ parser.mly exists │
│ command: menhir ... │ │
│ outputs: ${base}.ml │ │
│ │ │
▼ ▼ ▼
register_custom() get_generators("mly") run(dest, src, modName)
│ │ │
│ Stored in global │ Returns matching │ substitute_variables()
│ generator registry │ generators │ Execute via sh -c
│ │ │ Produce .ml/.mli
Custom generators defined in .obuild take precedence over built-in ones, allowing users to override default behavior (e.g., using menhir instead of ocamlyacc for .mly files).
The ppx_resolver.ml module handles resolution of PPX preprocessors and legacy camlp4 syntax extensions.
PPX and syntax packages are resolved through the META file system using special predicates:
For a target with syntax dependencies:
1. Collect syntax deps from project DAG
2. For each syntax dep:
├── Internal (project library)?
│ └── Use compiled bytecode archive path
└── External (findlib package)?
└── Query META with predicates:
[Syntax, Preprocessor, Camlp4o/Camlp4r]
3. Generate -I include paths
4. Generate archive file arguments
5. Return complete preprocessor command
get_syntax_pp: Generates preprocessor flags (-Ipaths and archive files) for syntax packagesget_target_pp: Resolves all syntax/preprocessor dependencies for a target and returns a complete preprocessor configuration
The resolver distinguishes between internal syntax packages (compiled as part of the project) and external ones (resolved via findlib META files).
Obuild supports configuration files (.obuildrc) for setting default values that apply before command-line argument parsing.
Configuration is loaded from two locations, in order:
~/.obuildrc— User-level defaults./.obuildrc— Project-level defaults
Project-level values override user-level values. Command-line arguments override both.
# Comments start with #
verbose = true
jobs = 4
ocamlopt = /usr/local/bin/ocamlopt
Each line is a key = value pair. Keys correspond to CLI option names (without the -- prefix).
The CLI framework (lib/base/cli.ml) loads configuration files via load_config() and applies them as argument defaults through run_with_config. This happens before command-line parsing, so CLI flags always take precedence.
Priority (highest to lowest):
1. Command-line arguments --jobs 8
2. Project .obuildrc jobs = 4
3. User ~/.obuildrc jobs = 2
4. Built-in defaults jobs = 2
| Command | Description |
|---|---|
configure |
Detect dependencies and prepare build configuration |
build |
Compile targets (all or specified) |
clean |
Remove build artifacts |
install |
Install compiled artifacts |
test |
Build and run test targets |
init |
Create a new project skeleton |
sdist |
Create a source distribution tarball |
get |
Query configuration values |
generate |
Generate helper files (see subcommands below) |
doc |
Generate documentation (not yet implemented) |
infer |
Infer project structure (not yet implemented) |
| Subcommand | Description |
|---|---|
generate merlin |
Create .merlin file for IDE support (source dirs, build dirs, packages) |
generate opam |
Create .opam file (OPAM 2.0 format) from project metadata |
generate completion |
Generate shell completion scripts (bash, zsh, fish) |
These options apply to all commands:
| Option | Short | Description |
|---|---|---|
--help |
-h |
Show help |
--version |
-V |
Show version |
--verbose |
-v |
Enable verbose output |
--quiet |
-q |
Suppress output |
--debug |
-d |
Enable debug output |
--debug+ |
Enable debug output with commands | |
--color |
Enable colored output | |
--findlib-conf |
Path to findlib configuration | |
--ocamlopt |
Path to ocamlopt compiler | |
--ocamldep |
Path to ocamldep tool | |
--ocamlc |
Path to ocamlc compiler | |
--cc |
Path to C compiler | |
--ar |
Path to ar archiver | |
--pkg-config |
Path to pkg-config tool | |
--ranlib |
Path to ranlib tool | |
--ocamldoc |
Path to ocamldoc tool | |
--ld |
Path to linker |
| Option | Default | Description |
|---|---|---|
--executable-native |
true | Build native executables |
--executable-bytecode |
false | Build bytecode executables |
--executable-profiling |
false | Build profiling executables |
--executable-debugging |
false | Build debugging executables |
--executable-as-obj |
false | Build executables as objects |
--library-native |
true | Build native libraries |
--library-bytecode |
true | Build bytecode libraries |
--library-profiling |
false | Build profiling libraries |
--library-debugging |
false | Build debugging libraries |
--library-plugin |
true (Unix) | Build library plugins |
--build-examples |
false | Build example targets |
--build-benchs |
false | Build benchmark targets |
--build-tests |
false | Build test targets |
--annot |
false | Generate annotation files |
| Option | Short | Description |
|---|---|---|
--jobs |
-j |
Number of parallel jobs (default: 2) |
--dot |
Dump dependency graph in DOT format | |
--noocamlmklib |
Disable ocamlmklib usage | |
-g |
Shorthand for --library-debugging --executable-debugging |
Trade-off: More complex implementation vs. better incrementality and parallelism.
Rationale: A single DAG cannot efficiently answer both "what changed?" (file-level) and "what order?" (task-level). Separating them allows:
- File DAG: Fine-grained mtime checks
- Steps DAG: Coarse-grained parallel scheduling
Trade-off: Callers must resolve subpackages vs. no double-resolution bugs.
Rationale: When the cache pre-resolved subpackages, callers that also resolved (common pattern) caused SubpackageNotFound errors. Returning root packages consistently prevents this class of bugs.
Trade-off: More verbose code vs. compile-time path safety.
Rationale: Path manipulation bugs are common and hard to debug. Type-safe filepath/filename types catch errors at compile time rather than runtime.
Trade-off: Less explicit control flow vs. simpler code structure.
Rationale: OCaml's exception model allows errors to propagate naturally without Result types threading through every function. The main entry point catches and formats all exceptions.
Trade-off: First build is slower vs. subsequent builds are faster.
Rationale: Parsing META files is I/O bound. Caching parsed results in memory dramatically speeds up dependency resolution for large projects with many dependencies.
Trade-off: More code, separate modules vs. cleaner separation of concerns.
Rationale: The new parser architecture (Lexer → Parser → AST → Validator) separates:
- Tokenization (obuild_lexer.ml)
- Syntax analysis (obuild_parser.ml)
- Data representation (obuild_ast.ml)
- Semantic validation (obuild_validate.ml)
This makes each component easier to test, modify, and understand.
Obuild's architecture reflects its goals of being declarative, parallel, and incremental:
┌─────────────────────────────────────────────────────────────────┐
│ ARCHITECTURE SUMMARY │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Declarative Input Parallel Execution │
│ │ │ │
│ .obuild file Scheduler + DAGs │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────┐ ┌─────────┐ │
│ │ Parsing │──────────────│ Build │ │
│ │ Layer │ Analysis │ Layer │ │
│ └─────────┘ └─────────┘ │
│ │ │ │
│ Project.t Compiled Artifacts │
│ │ │ │
│ Type-Safe Incremental Rebuilds │
│ Representation │
│ │
└─────────────────────────────────────────────────────────────────┘
The key insight is that build systems solve two distinct problems:
- What to build: Declarative configuration, dependency resolution
- How to build: Scheduling, parallelism, incrementality
Obuild addresses both through its layered architecture and dual-DAG design.
Obuild has been designed to be used as a library eventually. The code is shifting towards using pure structures and functions, so that things can be reused. There is some global state that will be eventually reduced to provide better control of each part.
One possible development would be to provide an optional daemon that monitors file changes and automatically rebuilds on demand without having to re-analyze the whole project.
Some other possible scenarios include having other programs use the project file format, either to provide tools to write them or tools that read them.