computer-graphics

Imported from wh1t3lord/computer-graphics (AGENTS.md). Install upstream with npx skills add wh1t3lord/computer-graphics. Copyright stays with the author.

AGENTS.md

Guidance for AI coding agents working in this repository.

Project overview

This is a personal, educational practice repository about AI & Computer Graphics (upstream: https://github.com/wh1t3lord/ai, author: wh1t3lord). Licensed under Apache 2.0. It is a collection of standalone, runnable demos — not a library, package, or deployable application. The README frames it as companion material for people learning slangpy.

The repository is organized into chapters, one per top-level directory:

• cg_raster/ — rasterization demos. This is the only chapter with actual code. It currently contains 19 demo scenes progressing from "draw a triangle" through textures, transforms/gimbal lock, Blinn-Phong lighting (ambient → diffuse → specular → materials → directional/point/spot lights), and a simplified linear render graph with a GPU resource manager (srm9). The README lists more planned topics (PBR, shadow mapping, voxel cone tracing, ReSTIR, neural shading, …) that are not implemented yet.
• cg_rtx/ — placeholder for a PBRT-style path tracer (only an empty readme.md).
• cg_neural/ — placeholder for Gaussian splatting / NeRF / diffusion-model demos (only an empty readme.md).
• data/ — runtime assets: shaders/raster/*.slang, textures/, models/ (empty; GLTF loading is a planned topic).

Tech stack

• Python 3.11 (README states >= 3.7, <= 3.12; the local .venv runs 3.11.9). No pyproject.toml/setup.py — plain scripts plus requirements.txt.
• slangpy 0.37.0 (import slangpy as spy) — the central framework. Provides the Vulkan spy.Device, spy.Window, render/compute pipelines, swapchain, the imgui-style spy.ui overlay, spy.math, and the runtime Slang shader compiler.
• Slang — all shaders are .slang files under data/shaders/raster/, compiled at runtime via device.load_program(path, ['mainVertex', 'mainPixel']).
• numpy — CPU-side vertex data and matrix math (view/model/projection matrices are built with numpy + spy.math helpers).
• pypng (import png) — PNG decoding in core/texture_naive.py.
• screeninfo — primary-monitor resolution detection for window creation.
• pyRenderdocApp — RenderDoc integration for GPU frame captures (see "RenderDoc" below).

Build and run commands

There is no build step, test suite, linter, formatter, or CI configuration in this repo.

# Setup (a .venv already exists in the repo root and is git-ignored)
pip install -r requirements.txt

# Run the raster demos (the only runnable chapter)
cd cg_raster
python main.py        # or from the repo root: python cg_raster/main.py

Notes:

• requirements.txt is saved as UTF-16 LE with BOM and CRLF line endings (pip reads it fine). If you edit it, keep the encoding or convert deliberately.
• The app opens a real window sized to the primary monitor and requires a Vulkan-capable GPU. It cannot run headless; spy.Device is created with enable_debug_layers=True and type=spy.DeviceType.vulkan.
• cg_raster/main.py imports core and scenes as top-level packages, so it must run with cg_raster/ as the working directory or as the script directory — it is not importable from the repo root as a module.
• Asset paths are derived from Path(__file__).parent.parent / 'data' in main.py, so they resolve regardless of the working directory.

Code organization and runtime architecture

Everything currently lives in cg_raster/:

• main.py — entry point. The App class registers every scene in an ordered self.scenes dict (name → instance), creates the window/device/UI, and runs the main loop: process events → maybe switch scene → scene.update(dt) → scene.render(). It also creates a shared core.ResourceManager in __register_scenes (initialized with 64 MB static buffer / 16 MB dynamic buffer / 256 MB texture budget) for scenes that want one — the current render-graph scenes (sr9*.py) do NOT use it; each builds a private ResourceManager sized by the SIZE_BUFFER_STATIC / SIZE_BUFFER_DYNAMIC / SIZE_TEXTURES variables in its own file. Users switch demos with the Left/Right arrow keys or the combobox in the imgui window.
• core/ — shared framework, re-exported as a flat API via core/__init__.py:
  • iscene.py — IScene, the base class every demo extends. Public methods (init, update, render, shutdown, on_resize, on_mouse_event, on_keyboard_event) are final dispatchers that call protected hooks (_init, _update, _render, _shutdown, _on_resize, _on_mouse_event, _on_keyboard_event). Scenes override only the underscore-prefixed hooks. init(...) receives (device, window, ui, ui_main_window, shaders_path, textures_path, models_path). RenderDoc frame capture is wired into IScene.render().
  • settings.py — global singleton g_Settings (currently just enable_renderdoc_capture, which main.py sets to True).
  • camera.py — Euler-based fly Camera (WASD + mouse look) that builds the view matrix with numpy. Input is optional (Camera(None) works), so a render pass can own a private camera. CameraQuat is an empty stub.
  • input.py — key-binding system: Input maps eInputBindingsType entries (move forward/back/left/right, camera pitch/yaw) to InputBindingState objects driven by spy.KeyboardEvent/spy.MouseEvent. Default movement keys are W/A/S/D; right mouse button toggles cursor capture.
  • model_naive.py — ModelNaive (vertex/index GPU buffers, TSR transform helpers) plus procedural geometry generators: unit box variants (position-only, +color, +uv, +normal) and a UV sphere.
  • texture_naive.py — TextureNaive: PNG → RGBA8 spy.Texture via pypng + numpy.
  • light_naive.py, material_naive.py — plain data classes for Blinn-Phong lights (ambient/directional/point/spot) and materials (color- or texture-based); their fields mirror the Slang-side structs.
  • offset_allocator.py — Python port of Sebastian Aaltonen's O(1) OffsetAllocator (256-bin freelist over a fixed byte range, neighbor merging) with allocation stats (get_stats, storage_report; note largest_free_region is rounded down to a bin size class, so it is a lower bound — used/free/counts are exact). Also has grow(new_size) (enlarges the range preserving existing allocations and neighbor-links the new tail so it merges later) and resize(new_size) (full reset at a new size).
  • buffer.py — Buffer: one big GPU buffer that hands out suballocations via an OffsetAllocator (its own instance when none is passed). eBufferType.kStatic = device_local memory (uploaded once, e.g. static geometry), eBufferType.kDynamic = upload memory (CPU writable). Writes go through CommandEncoder.upload_buffer_data; allocations are bound with {"buffer": ..., "offset": ...} pairs in set_render_state. reallocate(new_size) grows the GPU buffer preserving contents (device-wait, copy_buffer, allocator grow), recreate(new_size) recreates it empty (used to restore the configured size; falls back to reset() when unchanged).
  • resource_manager.py — ResourceManager(supports_reallocation=False): owns one static + one dynamic buffer (each with a shared allocator living on the manager), tracks uploaded textures against a byte budget, holds the named shared resources render passes produce/consume, reports GPU memory stats (get_gpu_stats_report — totals are the configured pool sizes, used is actual suballocated/registered bytes), and supports reset() (frees scene resources and restores the pool sizes configured in init(), recreating any pool that grew). Suballocations go through allocate_static/allocate_dynamic (+ free_static/free_dynamic): with supports_reallocation=False an exhausted pool returns None (textures raise), with True the pool is recreated bigger (doubling until the request fits, contents/offsets preserved; the texture budget just grows since textures are individual allocations). Direct buffer_static.allocate() calls bypass the reallocation mechanism.
  • render_graph.py — eRenderGraphMode (kEditor/kGame), IRenderPass (same dispatcher/hook pattern as IScene: _init, _update, _render, _shutdown), and LinearRenderGraph (passes added manually, executed in order, no input/output resolution — shared data goes through the resource manager). IRenderPass also carries: an enabled flag (the graph skips disabled passes in update/render — this is what the UI checkboxes toggle), an optional per-pass camera override (get_camera() falls back to the shared 'camera' resource when unset; a pass-owned camera is advanced by the pass dispatcher itself), resolution_scale + get_viewport_state(texture) for passes that render at a fraction of the target size, and enable_renderdoc_events — when True, the render dispatcher wraps _render() in a debug group named after the pass class (push_debug_group/pop_debug_group on the frame's command encoder, fetched per call, balanced via try/finally). This flag is injected, never read from globals by the pass: LinearRenderGraph(enable_renderdoc_events=...) takes it from the scene, add_pass stamps it onto each pass, and scenes pass core.g_Settings.enable_renderdoc_capture (see "RenderDoc").
• scenes/ — one file per demo, re-exported from scenes/__init__.py. Naming convention: sre.py (empty), srt*.py (triangle demos), srm<N>_cam.py (model demos in tutorial order), sr9*.py (grid/render-graph demos). Each scene class is named SceneRaster.... scenes/passes/ holds scene-independent render passes: currently DebugGridPass, an infinite debug grid (one fullscreen triangle, grid computed analytically in data/shaders/raster/passes/debug_grid_infinite.slang via inverse view-projection unprojection; the pass owns its program/pipeline and needs no vertex buffers or depth attachment), and DebugGridVariablePass, a sized variant (data/shaders/raster/passes/debug_grid_variable.slang, grid size 0 = 4x4, 1 = 16x16, 2 = 64x64, 3 = infinite) whose constructor takes static_mode: None compiles the shader with the DYNAMIC define and feeds the mode per frame from the grid_mode attribute through the tiny g_gridMode uniform, while 0..3 compiles with STATIC=<mode> and bakes one implementation at compile time (no uniform at all). Both variants are built in a dedicated slang session (see "Python ↔ Slang data conventions"). Pass shaders live under data/shaders/raster/passes/. sr9.py renders that grid through the linear render graph: editor mode registers DebugGridPass + UIPass (game mode is a stub). UIPass is defined in the scene file because its widget set is scene specific — it owns ALL imgui widgets of the scene (a 'camera' group and a 'render passes' group with one enable/disable checkbox per registered pass, labeled with the pass class name, built lazily on first update). The scene class creates no imgui widgets itself. The sr9*.py scenes take (eRenderGraphMode, ResourceManager) in their constructor but are registered in main.py without a manager, so each builds a private ResourceManager sized by the SIZE_BUFFER_STATIC / SIZE_BUFFER_DYNAMIC / SIZE_TEXTURES module variables in its own file (the 'gpu memory' UI totals are calculated from exactly these) with RESOURCE_MANAGER_SUPPORTS_REALLOCATION = False (see core/resource_manager.py). sr9_1.py/sr9_2.py follow the same architecture with DebugGridVariablePass: sr9_1.py uses static_mode=None (its UIPass adds a grid-size combobox that writes the pass's grid_mode attribute), sr9_2.py uses static_mode=GRID_MODE_STATIC (a module constant, no combobox on purpose); their UIPasses also show the resource manager's get_gpu_stats_report() in a 'gpu memory' group.
• scenes/ui/ — reusable editor UI, one module per concern, re-exported from scenes/ui/__init__.py. editor.py holds the whole editor state of a scene, ECS style: an entity is just an integer id (monotonic, never recycled), a component is a data dict attached to it under a type name (addable types live in COMPONENT_FACTORIES — only the placeholder tag exists for now), and EditorState tracks the current entity/component selection, keeping it consistent on deletions; widgets never own editor data, they only read/mutate this state. ui_widget_scene_objects.py — build_scene_objects_widget(parent, editor_state): entity list box + add/delete buttons (update() per frame, shutdown() when the pass dies). ui_widget_entity_components.py — build_entity_components_widget(ui, editor_state): window that appears while an entity is selected; component list with add/delete and the selected component's data section below a separator (per-type data builders in _data_section_builders, rebuilt only on selection change). Note: slangpy's ListBox renders no selection highlight (verified by pixel-diffing rendered frames in slangpy 0.37 — value/callback mechanics work, the highlight is simply not drawn), so both editor list boxes mark the selected row with a '> ' text prefix and rebuild items when the selection changes. sr9_3.py demonstrates the objects window, sr9_4.py adds the components window; both scenes create a fresh EditorState in _init and share it as the 'editor_state' resource.

Scene/shader pairing convention: a scene scenes/srm7_cam.py loads data/shaders/raster/srm7_cam.slang — same base name. Shaders declare mainVertex/mainPixel entry points with [shader("vertex")]/[shader("fragment")], plain uniform globals (e.g. uniform float4x4 g_mModel;), and structs for light/material data.

Per-frame rendering pattern inside a scene's _render(): create command encoder → swapchain.acquire_next_image() → clear → begin_render_pass → rp.bind_pipeline(...) → assign uniforms through spy.ShaderCursor → rp.set_render_state({viewports, scissor_rects, vertex_buffers, index_buffer, index_format}) → rp.draw_indexed(...) → render UI into the same texture → submit → present(). _shutdown() must call device.wait() before releasing the swapchain and GPU resources (scenes are shut down and re-initialized on every scene switch).

Python ↔ Slang data conventions

• Uniforms are written by name through the cursor: cursor.g_mModel = self.model.mModel; Slang structs are assigned as Python dicts whose snake_case keys match the Slang field names exactly (e.g. cursor.g_lightSpot = {"intensity_ambient": ..., "constants": np.array([...], dtype=np.float32)}).
• Vertex input layouts are declared in Python with device.create_input_layout using explicit byte offsets (float_size * 3, etc.) and semantics (POSITION, COLOR, NORMAL, TEXCOORD) that must match the Slang IN_VERTEX struct and the numpy vertex arrays (all np.float32, indices np.uint32 with spy.IndexFormat.uint32).
• Color targets are spy.Format.rgba32_float; textures are uploaded as rgba8_unorm.
• Per-program preprocessor defines (shader specialization) go through a dedicated slang session, because Device.load_program has no defines parameter and SlangCompilerOptions.defines on the device itself is session-wide: session = device.create_slang_session(compiler_options={'defines': {'STATIC': '1'}}) then session.load_program(path, ['mainVertex', 'mainPixel']) (see scenes/passes/debug_grid_variable.py; keep the session alive until shutdown, the linked program depends on it).

Code style guidelines

• Match the existing tutorial style: the code doubles as teaching material, so explanatory comments about graphics concepts (offset math, gimbal lock, GPU/CPU sync points) are intentional — keep them accurate when editing.
• snake_case everywhere; 4-space indentation; type annotations on function signatures written with spaces around the colon (width : int). Scene files start with an # Author: wh1t3lord header comment.
• The 'triangle, one vertex' balance rule: every piece of code should be equally easy to maintain, simple to read, and cheap in memory/execution — when one of the three would suffer disproportionately, rebalance instead of maximizing a single one. Concretely: no heavy abstractions for tiny demos (a dict-based ECS beats archetype tables at editor scale), but also no needless per-frame waste where a per-change update works (UI list items are rebuilt only when the underlying set changes, selections are stored as stable ids), and no micro-optimizations that make tutorial code unreadable.
• Coupling needs a justification: classes should know as little about each other as possible, and every connection between two pieces of code must have a one-sentence reason that survives the question "why does X know about Y?" — if it can't be justified, it doesn't exist. App-wide state (settings, device, per-frame resources) is injected by the owner/composition root, never pulled from globals by lower-level components: scenes pass g_Settings values down, the graph stamps enable_renderdoc_events onto passes, and a pass never reads g_Settings itself. Dependencies point toward the composition root, sideways only when the concept genuinely requires it.
• Shader authoring: target the portable common subset that compiles correctly across Slang's backends (no vendor-specific intrinsics or compiler quirks unless guarded and justified). Optimize by construction, not by obfuscation: compile-time specialization over runtime branching when a value is known at compile time (the STATIC/DYNAMIC grid variants), uniform traffic kept minimal and sized to what modern APIs move cheapest (push-constant-sized per-draw data), analytic computation over lookup textures where it's cheaper and simpler. Readability rules apply to shaders the same as to Python.
• Keep demos self-contained: each scene builds its own pipeline, buffers, and UI widgets inside _init and releases them in _shutdown. Shared functionality belongs in core/, and new scenes must be re-exported in scenes/__init__.py and registered in App.__register_scenes in main.py.
• Git history uses short imperative commit messages, mostly prefixed ADD: / FIX:-style.

Testing

There are no tests, no test framework, and no CI. Verification is manual: run python cg_raster/main.py, switch to the affected scene with the arrow keys or combobox, and check the rendered output. Because the app needs a display and a Vulkan GPU, automated verification from an agent session is usually not possible — say so instead of claiming a run succeeded.

RenderDoc

main.py sets core.g_Settings.enable_renderdoc_capture = True, and IScene loads RenderDoc via pyRenderdocApp.load_render_doc() at scene construction. This requires RenderDoc installed on the machine; when available, the UI shows the capture save directory. If RenderDoc is not installed, spy.renderdoc.is_available() gates the UI text, but expect load_render_doc() to be the fragile point on machines without it.

Separately, every render pass wrapped in a LinearRenderGraph can emit a named debug group around its render work (class name, API-level VK_EXT_debug_utils/PIX-style markers via CommandEncoder.push_debug_group — visible in RenderDoc captures and any other graphics debugger). The switch is the same app setting, but passes stay ignorant of globals: the scene injects it into LinearRenderGraph(enable_renderdoc_events=...), and add_pass stamps it onto each IRenderPass (see core/render_graph.py).

Security considerations

• Nothing network-facing: the app is a local windowed demo and loads only local assets from data/.
• Some spots use raise "string" / bare raise (e.g. main.py path checks, texture_naive.py) — existing style, but prefer proper exceptions in new code.
• requirements.txt pins exact versions; the checked-in .venv/ is git-ignored local state — never commit it.
• GPU shaders are compiled from data/shaders/ at runtime; treat shader files as executable code and don't introduce file loads from untrusted paths.