Imported from yona-lang/yona (
AGENTS.md). Install upstream withnpx skills add yona-lang/yona. Copyright stays with the author.
AGENTS.md
This file provides guidance to Codex (Codex.ai/code) when working with code in this repository.
Build Commands
Prerequisites
- LLVM 22+ (Fedora:
sudo dnf install llvm llvm-devel llvm-libs cmake ninja-build cli11-devel doctest-devel pcre2-devel lld-devel) - CMake 3.15+, Ninja
- C++23 capable compiler
- Optional —
Std\GpuVulkan discovery probe (Linux/macOS): Vulkan SDK / loader so CMake findsVulkan::Vulkan(Fedora:sudo dnf install vulkan-devel vulkan-loader-devel; macOS:brew install molten-vk vulkan-headers vulkan-loader). Without it, GPU counts stay at zero. After installing those packages, runcmake --preset …again; the configure log records the discovered Vulkan include and loader-library paths. On macOS the runtimedlopens Homebrew/SDKlibvulkan.1.dyliborlibMoltenVK.dyliband enables portability instance/device extensions. Metal usually has noshaderInt64;hasGpuis still 1 when the device is ready, and IntArray GPU kernels use i32 when values fit. User-facing compute (mapGpu,Promise, fences) is specified indocs/design-gpu-async.md.
Configure and build (Linux)
cmake --preset x64-debug-linux
cmake --build --preset build-debug-linux
Replace linux with macos or omit the suffix for Windows. Replace debug with release for release builds.
Optional GPU: add -DYONA_ENABLE_VULKAN=ON and set VULKAN_SDK to the LunarG root (macOS: brew install molten-vk vulkan-headers vulkan-loader is enough; CMake uses HOMEBREW_PREFIX or brew --prefix) so CMake builds the canonical yona_runtime archive with Vulkan support (see docs/gpu-architecture.md). Default builds do not require it. Reconfigure and rebuild after changing this option; installed tools never compile runtime sources. With Vulkan enabled at configure time, Std\Gpu.vulkanStatus may return vulkan-device after runtime dlopen/LoadLibrary dispatch creates a compute queue. Metal usually has no shaderInt64; hasGpu is 1 when the device is ready and IntArray GPU kernels use i32 when values fit.
Compiler flags
-O0to-O3— optimization level (default O2)-g— emit DWARF debug info--Wall,--Wextra,--Werror,-w— warning control--emit-ir— print LLVM IR instead of compiling--emit-obj— emit object file only (don't link)-I path— module search paths for.yonaifiles--explain E0100— show detailed explanation for an error code
Prelude
Available in all programs without imports (from lib/Prelude.yona):
Types: Linear a, Option a (Some/None), Result a e (Ok/Err), Iterator a
Functions: identity, const, flip, compose
foldl, foldr, map, and filter are not prelude — import them from Std\List.
Adding to the prelude (unified — one source of truth):
- For native functions: add the implementation to its owning
src/Runtime/component and add theFNline tolib/Prelude.yonai - For Yona functions: edit
lib/Prelude.yona, recompile withyonac lib/Prelude.yona, move.yonaitolib/ - No manual registration needed —
semantics::InterfaceCatalogreads.yonaiand installs parser/type-checker contracts;Codegen::loadPrelude()loads only LLVM lowering metadata.
Run tests
# Via CTest
ctest --preset unit-tests-linux
# Directly
./out/build/x64-debug-linux/tests
# Run a specific test (doctest subcase filter)
./out/build/x64-debug-linux/tests -tc="TestName"
On Windows, set YONA_PATH to …/test/code;…/lib and YONAC_CC to the full path of clang.exe from the same complete clang+llvm-*-windows-msvc tree as LLVM_INSTALL_PREFIX (or put that bin on PATH). CI appends C:\LLVM\bin to PATH and sets YONAC_CC for the same reason. See INSTALL.md (Windows): Complete Windows LLVM tree and YONAC_CC and doctest — incomplete LLVM installs (LLVMExports.cmake missing LTO.lib, llvm-ar.exe, etc.) break find_package(LLVM); LLVM_INSTALL_PREFIX must not contain typos.
When CMake is configured with -DYONA_ENABLE_VULKAN=ON, CTest also registers
doctest_gpu_vulkan, which runs tests -tc=*gpu?vulkan* against the same
Vulkan-enabled yona_runtime archive used by generated programs. Example:
ctest --test-dir out/build/x64-debug -R doctest_gpu_vulkan -V (adjust the
build directory for your preset).
Vulkan device selection, opt-in env vars, and diagnostics are documented in
docs/gpu-architecture.md and docs/api/Gpu.md (YONA_GPU_VULKAN_PHYSICAL_DEVICE_INDEX,
YonaRuntimeGpuVulkanDeviceLastNote(), Std\Gpu.vulkanLastNote).
Format code
./scripts/format.sh # runs clang-format on include/, src/, test/, cli/
Generate API docs
python3 scripts/gendocs.py # extracts ## comments → docs/api/
Architecture Overview
Yona language compiler using LLVM. Pipeline: source management → lexer/parser → AST → semantic model → typed IR → LLVM IR/object generation → toolchain link.
Key Design Patterns
Newline-aware lexer: Newlines are significant tokens (YNEWLINE) that delimit expressions in case arms, do-blocks, and module function bodies. Semicolons are equivalent to newlines. Inside brackets ((), [], {}), newlines are suppressed (treated as whitespace). After binary operators and continuation tokens (->, =, ,), the following newline is suppressed to allow natural line continuation. This enables juxtaposition-based function application (f x y instead of requiring f(x, y)) without ambiguity at expression boundaries.
Visitor pattern with generic return types: AstVisitor<ResultType> is a templated base class. AST nodes implement accept() that dispatches to the correct visit() overload.
LLVM Codegen (src/Codegen/Codegen.cpp): Type-directed code generation using TypedValue = {Value*, CType}. Every codegen method returns both an LLVM value and its type tag. Functions are compiled with deferred compilation — stored as AST at definition, compiled at call site where argument types are known (monomorphization). Closures use env-passing convention: {fn_ptr, ret_tag, arity, num_caps, heap_mask, cap0, ...} heap arrays, functions take (ptr env, args...). The yonac CLI compiles Yona source to native executables via LLVM.
Algebraic Data Types: type Option a = Some a | None — named constructors with typed fields. ADT fields support function type signatures: type Stream a = Next a (() -> Stream a) | End. Constructors are first-class functions. Non-recursive ADTs use flat structs {i64 tag, payload}; recursive ADTs and ADTs with function fields are heap-allocated.
Module System: Modules are top-level declarations (ModuleDecl), not expressions. No as/end — modules end at EOF. export statements: export func, export type Name, export f from Mod. Module-level extern declarations for C bindings (extern sqrt : Float -> Float). Compile to native object files with C-ABI exports. Interface files (.yonai) provide cross-module type metadata. Exported functions include source text in .yonai (GENFN_BEGIN/GENFN_END) for cross-module monomorphization — when call-site types differ from the pre-compiled signature, the source is re-parsed and compiled locally with actual types.
Memory Management: Atomic reference counting with recursive destructors and Perceus-linear callee-owns ABI. See docs/memory-management.md for full details. Key points: All heap objects use YonaRuntimeAllocate with a [refcount, type_tag_encoded] header. Atomic rc_inc/rc_dec (__atomic_fetch_add/sub). Recursive destructors free children via heap_mask bitmasks (closures, ADTs, tuples, seqs, sets, dicts). Unified Perceus: all heap types follow callee-owns at call sites — single-use args transferred without DUP, runtime consume paths rc_dec on path-copy. Per-branch transferred_seqs_ / transferred_maps_ scoping in if-expressions and case arms handles asymmetric transfers with SSA dominance preserved via pre_blocks snapshotting. Unique-owner in-place optimization (rc==1) for seq cons/tail and HAMT put. Weak self-references break recursive closure cycles. io_uring buffer pinning for async I/O safety. Escape analysis (include/yona/Codegen/EscapeAnalysis.h) for arena allocation of non-escaping values.
Symbol interning: Symbols (:ok, :none) are interned to i64 IDs at compile time. Comparison is icmp eq i64 (1 cycle). Pattern matching on symbols generates integer switch/select.
Core Components
- AST (
include/yona/Syntax/Ast.h): Node hierarchy rooted atAstNode. Major branches:ExprNode(expressions),PatternNode(pattern matching),ScopedNode(scope-creating). Each node carries aSourceRangeresolved through its parsed source manager. - Codegen (
src/Codegen/Codegen.cpp): LLVM IR generation withTypedValuesystem. Supports literals, arithmetic, functions, closures, recursion, case expressions, tuples, sequences, symbols, sets, dicts, ADTs, or-patterns, higher-order functions, partial application, generators/comprehensions. Runtime components compile once into theyona_runtimearchive. - Type System (
include/yona/Model/Types.h): Variant-based types including builtins, function types, product/sum types, record types, ADTs, and named types. The codegen usesCType+TypedValuewith compile-time type inference and monomorphization. - Pattern Matching:
CaseExprcontains a target expression and vector ofCaseClause(pattern, body). Pattern types include value, tuple, seq, head-tail ([h|t]), dict, record, constructor (Some x),asbinding (@), and or-patterns. - Module System: FQN-based. Modules (
ModuleDeclin AST) are top-level declarations, not expressions. They compile to object files with C-ABI exports via name mangling (YonaPkgModFunc). Interface files (.yonai) for cross-module type-safe linking. Three import styles: selective, wildcard, FQN calls (Mod::func). Cross-module generics via GENFN source in.yonai— on-demand re-parse and monomorphization when call-site types differ. Trait instance methods haveExternalLinkagefor cross-module trait dispatch.
Build Targets
yona_support,yona_model,yona_syntax,yona_interface,yona_semantics,yona_typed_ir,yona_codegen_llvm, andyona_toolchain: Acyclic compiler components.yona_typed_coreandyona_lsp: Canonical semantic adapters.yona_lib(shared) /yona_lib_static: Aggregate compiler libraries built from the component object targets.yonac: Compiler executable (links the aggregate compiler library + CLI11)yona: Yona-written runner (tools/yona/main.yona) — shebang /-e/ stdinyona-repl: C++ REPL executable; started byyonaon a TTYyona_runtime: Canonical aggregate runtime archive linked into generated programstests: Test executable (linksyona_lib_static+ doctest)
Dependencies
- LLVM 22+: Code generation backend
- CLI11: Command-line parsing — system package (
cli11-devel/libcli11-dev/ Homebrewcli11); FetchContent only if-DYONA_FETCH_DEPS=ON(default) and no package is found - doctest: Test framework — system package (
doctest-devel/doctest-dev/ Homebrewdoctest); same FetchContent fallback. Distro/Homebrew builds pass-DYONA_FETCH_DEPS=OFFand-DBUILD_TESTING=OFF.
Yona Idioms (IMPORTANT — follow these when writing Yona code)
- Don't nest let expressions. Use multi-binding:
let x = 1, y = 2 in x + y letanddohave different semantics.letbinds values (independent RHSs may parallelize);dosequences effects. Combining them is valid when you need both (let a = …, b = … in do … end). Do not uselet _ = effectto sequence, and do not wrap a single expression indo. Usedo ... endfor ordered side effects- Use comma-separated imports:
import a from X, b from Y in ... - Use
withfor resources:with h = openFile "f" Read in ... end - Use parallel comprehensions:
[| f x for x = xs ]for concurrent processing - Use
Std\List.foldlfor aggregation:import foldl from Std\List in foldl (\a b -> a + b) 0 xs(tail-recursive, compiled to a loop) - Use iterators for streaming:
readLines,chars,splitreturnIterator(O(1) memory) - Prelude needs no import:
Some,None,Ok,Err,Linear,Iterator,identity,const,flip,compose - See
docs/style-guide.mdfor the full guide
Stdlib implementation rule (IMPORTANT)
Anything that can be written in Yona must be written in Yona. Drop to C
only when there is no way to express the operation in pure Yona — typically
because it needs an OS syscall (file I/O, network, processes, time, signals),
mutable state primitives the language doesn't expose (atomics, locks, channel
buffers), bit-level layout control (byte arrays, hashing, crypto), an
external C library binding (PCRE2, OpenSSL, libxml2), or performance-critical
hot loops with measured wins (matrix kernels, codec inner loops). Pure data
transformations, pattern matching, recursion, and combinator plumbing all
belong in .yona files. The C runtime should be the substrate, not the
default. When in doubt: write the Yona version first, profile if needed, and
only then consider lowering to C.
Bug-tracking rule (IMPORTANT)
Whenever you discover a bug — parser, codegen, runtime, anything — append
it to docs/todo-list.md immediately, with a one-line repro, and stop to
ask which bug(s) to fix. Working around a bug silently buries the
information; a bug list with reproductions accumulates the data we need to
prioritize compiler work. Don't keep coding past a fresh bug discovery
without first noting it and checking which one to attack next.
Keep docs up to date
When finishing a feature or bugfix, update the matching plan under
docs/superpowers/plans/, docs/todo-list.md, and CHANGELOG.md in the
same change. Use Unreleased in the changelog if VERSION is unchanged.
Fix feature docs (docs/*.md) when they would otherwise contradict the code.
Also update the public site (site/src/content/docs/) whenever
user-facing language, CLI, prelude, or stdlib behavior changes. Learn /
Guides / Reference pages there are handwritten and are the published source
of truth — do not leave them stale because an internal doc already exists.
After lib/Std/*.yona API comment changes, run python3 scripts/gendocs.py.
Preview with cd site && pnpm dev.
Verify examples against yonac, not folklore.
Do not silently leave stale checkboxes.
Development Workflow
- New language features require changes across: Lexer → Parser → AST → Codegen
- AST modifications must update
include/yona/Syntax/Ast.h,include/yona/Syntax/AstVisitor.h, and codegen. - Tests are in
test/— codegen E2E fixtures intest/Fixtures/Codegen/*.yona+*.expected - ADT tests in
test/Codegen/AdtTest.cpp, trait/cross-module tests intest/Semantics/TraitTest.cpp - Stdlib in
lib/Std/— 26 modules: 12 pure Yona + 14 C runtime (see docs/todo-list.md for full list) - Stdlib fixture tests in
test/Fixtures/Codegen/stdlib_*.yona+*.expected - Platform-specific runtime in
src/Runtime/Platform/—include/yona/Runtime/Platform/IoUring.h(shared io_uring, Linux),include/yona/Runtime/Platform/Api.h,FileLinux.c,NetLinux.c, andOsLinux.c - API docs generated by
python3 scripts/gendocs.py→docs/api/
