XLion_Screenshot

Imported from Zhen-Xing-Shi/XLion_Screenshot (AGENTS.md). Install upstream with npx skills add Zhen-Xing-Shi/XLion_Screenshot. Copyright stays with the author.

AGENTS.md - xlion_screenshot

Screenshot tool for Ubuntu 26.04 Wayland on GNOME/Mutter. Runtime is Python 3.14 with GTK4/PyGObject, cairo, Rsvg/librsvg, Pillow, OpenCV, Gio D-Bus, StatusNotifier/AppIndicator tray registration, and a bundled GNOME Shell helper extension.

Setup

sudo apt install python3-gi-cairo gir1.2-rsvg-2.0 python3-opencv gnome-shell-ubuntu-extensions

python3-gi-cairo is required for GTK draw callbacks that receive cairo.Context; it is not a pip package. gir1.2-rsvg-2.0 is used to render toolbar SVG icons from icons/ into the Cairo toolbar. python3-opencv provides the fast native visual-region segmentation backend; without it, region detection falls back to the slower pure Python/Pillow implementation. gnome-shell-ubuntu-extensions provides Ubuntu's AppIndicators host extension used to display the app process StatusNotifier tray icon. This project assumes dependencies are provided by the system Python packages. Do not add a virtualenv or pip-managed dependency path unless the user explicitly asks for a packaging change.

Running

./xlion_screenshot
packaging/build_deb.sh
sudo apt install ./build/deb/xlion-screenshot_<version>_amd64.deb
xlion_screenshot --daemon

The deb package installs /usr/bin/xlion_screenshot, installs the system GNOME Shell extension at:

/usr/share/gnome-shell/extensions/xlion-screenshot-window-info-v4@xlion-screenshot.local

The package post-install script also copies the helper extension into the desktop user's local extension directory when needed and tries to enable it via gnome-extensions or GNOME Shell D-Bus. If window selection still falls back to fullscreen after install, log out and back in, then verify:

gnome-extensions enable ubuntu-appindicators@ubuntu.com
gnome-extensions enable xlion-screenshot-window-info-v4@xlion-screenshot.local

Testing

python3 -m pytest tests/ -q
python3 -m pytest tests/test_editor.py -v
python3 -m pytest tests/test_window_detection.py -v
python3 -m pytest tests/test_visual_regions.py -v

Current baseline: 319 passed, 1 skipped. The skipped live test is tests/test_screenshot.py::TestFallbackChain::test_default_methods_are_called; it requires a real GNOME desktop session with screenshot permission.

Debian Packaging

The deb builder writes packages under build/deb/. Normal packaging uses the committed runtime snapshot at:

third_party/runtime/ubuntu-26.04-amd64

The normal rebuild command is:

packaging/build_deb.sh

The output is:

build/deb/xlion-screenshot_<version>_amd64.deb

The package version is read only from root version.txt; do not pass VERSION=... to the build script. The deb builder verifies third_party/runtime/ubuntu-26.04-amd64/manifest.sha256 and must not collect Python, OpenCV, GTK, Rsvg, or bundled native libraries from the host.

Only when intentionally refreshing the committed runtime snapshot, run:

packaging/build_runtime_bundle.sh

That script reuses build/deb/opencv-minimal-install when it exists. If the prebuilt OpenCV directory is missing or incomplete, set OPENCV_PREBUILT_DIR=/path/to/opencv-minimal-install or OPENCV_SOURCE_DIR=/path/to/opencv for packaging/build_runtime_bundle.sh.

Architecture

xlion_screenshot (entry script)
  -> main.py (Gtk.Application launcher, D-Bus daemon, StatusNotifier tray)
     -> selector.py (fullscreen overlay, auto/window/region selection)
        -> screenshot.py (fullscreen capture backend chain)
        -> visual_regions.py (Pillow visual block detection inside windows)
        -> editor.py (annotation UI after selection)
           -> annotations.py (Pillow Annotation model + final rendering)
           -> notify.py (FreeDesktop Notifications D-Bus wrapper)

gnome-extension/xlion-screenshot-window-info-v4@xlion-screenshot.local/
  -> extension.js (D-Bus window frame rectangles, Shell screenshot, clipboard,
                   shortcut/config helpers)
  -> metadata.json (GNOME Shell 50 extension metadata)

Default startup uses selector.run_selector(app), which launches auto mode:

• click selects the hovered visual region inside a window when available;
• if no internal visual region is found, click selects the hovered window;
• if no window is hovered, click selects fullscreen;
• drag past AUTO_REGION_DRAG_THRESHOLD switches to manual region selection;
• run_window_selector() and run_region_selector() still exist for tests and explicit mode entry points.

Capture Backend

The tray icon is not a GNOME Shell PanelMenu; the Python daemon exports org.kde.StatusNotifierItem at /StatusNotifierItem and a minimal com.canonical.dbusmenu menu at /StatusNotifierMenu. Ubuntu's already-loaded ubuntu-appindicators@ubuntu.com extension displays it.

The normal tray/shortcut path asks the bundled GNOME Shell helper extension to capture pixels with Shell.Screenshot() and pass the temporary PNG to the Python daemon. If the helper extension is not loaded, the tray screenshot action falls back to selector.run_selector() and screenshot.capture_fullscreen(), which only uses the org.gnome.Shell.Screenshot D-Bus API.

Each backend returns a copied PIL.Image and removes any temporary PNG it created. Tests inject capture_methods; keep that seam stable.

Window Detection

selector.get_windows() tries these methods in order:

1. bundled GNOME Shell extension D-Bus API: org.xlion_screenshot.WindowInfo.ListWindows;
2. org.gnome.Shell.Introspect.GetWindows;
3. legacy org.gnome.Shell.Eval.

Window rectangles from GNOME Shell are logical coordinates. Screenshot pixels may be scaled. Hover hit-testing uses the unscaled mouse position, then selected rectangles are converted to image coordinates with scale_x and scale_y.

Visual Region Detection

After selector.find_window_at_position() finds the hovered window, selector.resolve_hover_selection() may call visual_regions.VisualRegionFinder with image-coordinate window and mouse rectangles. The visual region detector uses only the captured screenshot pixels. It does not inspect browser DOM, accessibility trees, or application internals.

VisualRegionFinder.region_at() looks for balanced, control-first, axis-aligned visual blocks. The OpenCV backend combines color-connected components for cards/sidebars/content panels with edge/contour candidates for input fields, buttons, and line-delimited subregions. It still filters tiny icons, pure text strokes, and decorative lines, then returns the smallest meaningful region under the pointer. If no credible region is found, the selector falls back to the whole window. The overlay precomputes one VisualRegionIndex per image/window rectangle and hover only does point lookup against that index. The OpenCV precompute path builds coarse structural regions from vectorized geometry, divider, and major quantized-color passes instead of probing a dense hover grid or per-text colors. Large windows are analyzed on a downsampled crop and mapped back to source image coordinates. Word-sized text, input fields, button-height controls, and tiny icons are intentionally ignored as independent screenshot regions; hover over those falls through to the containing panel or window. Duplicate window rectangles reuse the same precomputed index. The selector does not run a hover-time visual-region fallback; missed regions remain missed until the next screenshot/precompute cycle.

Editor Behavior

Tools are defined in editor.TOOLS: rect, arrow, text, brush, mosaic, undo, copy, save.

• Rectangles, arrows, text, brush, and mosaic annotations are stored as Annotation instances.
• Toolbar buttons render the matching SVG files from icons/ through Rsvg into the existing Cairo drawing context.
• Final output renders annotations onto the full screenshot first, then crops to the selected rectangle unless the selection is fullscreen.
• Copy writes the final PNG to a temporary file and asks the daemon over D-Bus to forward it to the GNOME Shell extension. The extension uses St.Clipboard.set_content(St.ClipboardType.CLIPBOARD, 'image/png', bytes) so GNOME Shell owns the clipboard payload after the editor exits. Do not move clipboard ownership into the daemon with Gdk.Clipboard; on Wayland, a non-interactive daemon can report success without publishing a usable selection to other applications.
• Save writes <configured_save_directory>/<unix_timestamp>.png. The default configured save directory is the user's XDG Pictures directory, falling back to ~/Pictures; the deb post-install settings initialization creates it when needed.
• The tray configuration is stored in ~/.config/xlion_screenshot/settings.ini; the Python daemon handles the StatusNotifier tray UI and configuration window, while the GNOME Shell extension handles the global screenshot shortcut and reloads settings through D-Bus when available. Python reads the save directory for editor saves.
• Notifications go through the session D-Bus org.freedesktop.Notifications.Notify API; notification failures must not crash the application.
• Pressing Escape quits in both selector and editor windows.

Critical Gotchas

Import order matters. In selector.py and editor.py, import cairo must stay before import gi. Otherwise GTK set_draw_func callbacks can fail with TypeError: Couldn't find foreign struct converter for 'cairo.Context'.

Capture before showing overlays. SelectorOverlay.__init__() must call self._load_screenshot() before self.show(). Otherwise the fullscreen overlay appears in its own screenshot.

Cairo scaling has two directions.

• Widget/mouse coords -> image coords: multiply by scale_x and scale_y.
• Image coords -> widget drawing: call cr.scale(1 / scale_x, 1 / scale_y).

Do not collapse this into a single scalar; mixed-DPI or nonmatching allocation sizes need independent X/Y factors.

Visual region coordinates are image coordinates. VisualRegionFinder expects window_rect and mouse_point after applying scale_x and scale_y. Do not feed widget/logical coordinates into visual_regions.py.

Visual detection is balanced control-first. Input fields, buttons, and obvious line-delimited subregions should be selectable as independent regions. Tiny icons, pure text strokes, and decorative lines should still fall back to a meaningful parent region. Keep the panel thresholds and control-specific thresholds separate so making controls selectable does not make every text row a candidate.

Cached parent regions must not hide child regions. A finder can cache both a parent panel and nested cards. region_at() must continue probing for smaller regions even when a cached larger region already contains the mouse.

Auto-mode drag suppression matters. In selector auto mode, a drag that crosses AUTO_REGION_DRAG_THRESHOLD sets _suppress_click so the release event does not also trigger window/fullscreen selection.

Clipboard must stay GTK-native. Do not replace it with wl-copy; it is not a project dependency and is less reliable here.

Text input and rendering are separate. Toolbar labels are drawn with PangoCairo. Final annotation text is rendered by Pillow in annotations.py, and current key handling only accepts printable ASCII keyvals.