Rust TUI Pixel Graphics: State of the Art (Mid-2026)#

Research report on high-fidelity pixel graphics in Rust terminal user interfaces, focused on rendering a cellular automata (falling-sand style) simulation with semantic cell types (flowing water, growing plants) in a terminal.


1. Comparison Table of Rendering Approaches#

Approach

Effective Pixels per Cell

Colors per Cell

Color Depth

Terminal Compatibility

Performance (200x80 term)

Complexity

Braille Canvas (U+2800-U+28FF)

2x4 = 8 dots

1 (fg only)

256 / TrueColor

Universal (Unicode)

Very fast; ~16K cells/frame

Low

Half-Block (▀▄█)

1x2 = 2 pixels

2 (fg + bg)

256 / TrueColor

Universal (Unicode)

Very fast; ~16K cells/frame

Low

Quadrant (▌▐▞▛)

2x2 = 4 pixels

1 (fg only)

256 / TrueColor

Good (Unicode)

Fast

Low-Med

Sextant (🬪🬫🬬)

2x3 = 6 pixels

1 (fg only)

256 / TrueColor

Moderate (newer Unicode)

Fast

Low-Med

Octant (𜶟𜶠)

2x4 = 8 pixels

1 (fg only)

256 / TrueColor

Low (rare Unicode support)

Fast

Med

Full Char Grid (1 char = 1 cell)

1x1 = 1 pixel

2 (fg + bg)

256 / TrueColor

Universal

Fastest

Lowest

Image Protocol (Kitty)

True pixel resolution

True 24-bit

24-bit

Kitty, WezTerm, Ghostty

Encoding overhead; fast display

High

Image Protocol (Sixel)

True pixel resolution

256-color palette

256 (typical)

xterm, foot, WezTerm, mintty

Moderate encoding overhead

High

Image Protocol (iTerm2)

True pixel resolution

True 24-bit

24-bit

iTerm2, WezTerm, Hyper, VSCode

Good

High

Hybrid (half-block grid + image protocol fallback)

1x2 or true pixel

2 or 24-bit

varies

Universal (graceful degradation)

Best of both

Med-High

Key Insight on Color Limitations#

The critical distinction for a cellular automata with semantic cell types:

  • Braille / Quadrant / Sextant / Octant (PatternGrid family): Only one foreground color per character cell. All 2x4 (or 2x2, 2x3) sub-dots share the same color. This means you cannot have a red sand dot and a blue water dot in the same character cell. This is a fundamental limitation of the Unicode pattern approach.

  • Half-Block: Two colors per cell (foreground for the top half, background for the bottom half). This is the key advantage for cellular automata where adjacent cells may have different materials.

  • Full char grid: Two colors per cell (fg + bg), but only 1x1 resolution. You can use the background color as the “cell color” and the character as a semantic symbol (e.g., ~ for water, # for sand, * for fire).

  • Image protocols: Unlimited colors at true pixel resolution, but require terminal support and have encoding overhead.

Sources:



3. Key Rust Crates and Their Maturity#

Crate

Version (Jul 2026)

Maturity

Purpose

Notes

ratatui

0.30.2

High (very active, modularized in 0.30)

Main TUI framework

Immediate-mode, double-buffer diffing, crossterm/termion/termwiz backends. no_std support. 100% documented.

ratatui-core

0.1.2

High

Core types for widget authors

Split out in 0.30 for API stability

ratatui-widgets

0.3.2

High

Widget library

Canvas, Chart, BarChart, etc.

ratatui-crossterm

0.1.2

High

Crossterm backend

Default backend for most apps

ratatui-image

11.0.6

High (394 commits, 390 dependents)

Image rendering widget

Sixel, Kitty, iTerm2, halfblocks. Screenshot tests in CI. Used by yazi, iamb, joshuto.

viuer

~1.0

Medium (less active)

Terminal image display

Kitty, iTerm2, Sixel, halfblocks. “Dumps” images (harder for TUI integration). CLI tool viu wraps it.

crossterm

0.29

High

Terminal I/O backend

Raw mode, event polling, ANSI escape. Used by ratatui by default.

image

latest

High (industry standard)

Image decoding/encoding

Used by ratatui-image for format support

tiny-skia

latest

High (linebender org)

CPU 2D rendering

Skia subset in pure Rust. Fastest software renderer. Useful for offscreen render-to-image approach.

rasterm

latest

Medium

Image protocol encoder

Encode to iTerm2/Kitty/Sixel escape sequences

palette

0.7.6

High

Color science library

Used by ratatui for color conversions. Useful for HSV/RGB color mapping.

Crate Recommendations for the Project#

Essential:

  • ratatui 0.30+ (framework)

  • ratatui-crossterm or crossterm (backend)

  • crossterm (event polling for input)

For the simulation:

  • Direct Buffer manipulation (no extra crates needed for half-block rendering)

  • palette crate for color space conversions (HSV gradients for water depth, fire temperature, etc.)

Optional (for image protocol fallback):

  • ratatui-image if you want to support true-pixel rendering on capable terminals

  • image crate as a dependency of ratatui-image

Not recommended for this use case:

  • viuer (designed for CLI image display, not TUI widget integration)

  • tiny-skia (overkill for blocky cellular automata; better for smooth 2D graphics)

Sources:


4. Existing Terminal Cellular Automata Projects#

Terminal-Based#

Project

Language

Renderer

Modes

Relevance

changkun/cellular-automaton-sandbox

Python

curses

27 modes including Falling Sand, Game of Life, Fluid Dynamics (LBM), Reaction-Diffusion, Forest Fire, DLA, Ising Model, Boids, etc.

Most directly relevant. Terminal-based falling sand with multiple materials (sand, water, fire). Uses curses for rendering. Python 3.12+, MIT license.

andrewosh/cellarium

(unknown)

(unknown)

Cellular automata playground

General CA playground

Non-Terminal (Rust, for algorithm reference)#

Project

Language

Renderer

Notes

PieKing1215/FallingSandEngine

Rust

(GPU/native)

Experimental 2D falling sand engine in Rust

ARez2/sandengine

Rust + GLSL

GPU shaders

Falling sand simulation engine with GLSL

wg-romank/sands-of-rust

Rust

WebGL

GPU-based particle simulation

GelamiSalami/GPU-Falling-Sand-CA

Rust

GPU

Block cellular automata on GPU

tranma/falling-sand-game

Rust

(native)

Cellular automata style falling sand

Key Finding#

No existing Rust-based terminal falling-sand simulation was found. The closest is changkun/cellular-automaton-sandbox (Python/curses), which proves the concept works in a terminal. The Rust falling-sand projects all use GPU rendering (wgpu/WebGL/shaders), not terminal output.

This means the project would be novel in the Rust TUI ecosystem.

Sources:


5. Performance Expectations and Bottlenecks#

Terminal Throughput Benchmarks#

From Kitty’s own benchmarks (megabytes/sec processed):

Terminal

ASCII

Unicode

CSI (escape codes)

Images

kitty 0.33

121.8

105.0

59.8

251.6

gnome-terminal

33.4

55.0

16.1

142.8

alacritty 0.13

43.1

46.5

32.5

94.1

wezterm

16.4

26.0

11.1

140.5

xterm

47.7

18.3

0.6

56.3

Key takeaway: Kitty is 2x faster than the next best terminal, especially for CSI (escape code) processing. For a TUI that sends many color changes per frame, CSI throughput matters most.

Ratatui’s Rendering Pipeline#

  1. App calls terminal.draw(|frame| ...)

  2. Widgets render to an intermediate Buffer (in-memory grid of Cells)

  3. After the closure returns, Terminal::flush() calculates a diff between current and previous buffer

  4. Only changed cells are written to the terminal as ANSI escape sequences

  5. Buffers are swapped; current buffer is wiped for next frame

This double-buffer diffing is the key optimization: if only 500 cells changed between frames (e.g., particles moved), only 500 cells worth of escape sequences are sent, not 16,000.

Performance Math for 200x80 Terminal#

Scenario

Cells Updated

Bytes per Cell (est.)

Total Bytes

Time at 60 MB/s CSI

Full screen redraw

16,000

~20 (pos + 2 colors + char)

~320 KB

~5 ms

Full screen (half-block, 2 colors)

16,000

~25

~400 KB

~6.7 ms

Partial update (10% changed)

1,600

~25

~40 KB

~0.7 ms

Full screen Braille

16,000

~15 (1 color + braille char)

~240 KB

~4 ms

At 30 FPS (33ms budget), even a full screen redraw at ~7ms leaves 26ms for simulation logic. At 60 FPS (16ms budget), a full redraw at ~7ms leaves 9ms for simulation. This is very feasible.

Bottlenecks#

  1. ANSI escape sequence overhead: Each cell change requires cursor positioning (\x1b[row;colH), color setting (\x1b[38;2;r;g;bm for fg, \x1b[48;2;r;g;bm for bg), and the character itself. With TrueColor, that is ~20-30 bytes per changed cell. Ratatui optimizes by batching adjacent changes and skipping unchanged cells.

  2. Terminal parsing speed: The terminal must parse the byte stream. Kitty leads at ~60 MB/s CSI; slower terminals like xterm manage <1 MB/s CSI. This is rarely the bottleneck for 200x80 grids.

  3. Terminal rendering (rasterization): GPU-accelerated terminals (Kitty, WezTerm, Alacritty, Ghostty) cache glyph atlases in GPU memory, so rendering 16K cells per frame is trivial. CPU-based terminals may struggle with full-screen updates at 60fps.

  4. Application-side buffer diffing: Ratatui’s diff algorithm compares 16K cells per frame. This is O(n) and takes microseconds on modern hardware. Not a bottleneck.

  5. Image protocol encoding (if using image approach): Sixel/Kitty encoding of a full-screen image can take 10-50ms depending on image size and color depth. This is the main argument against the image protocol approach for real-time simulation.

Optimization Tips#

  • Use Color::Ansi256(n) instead of Color::Rgb(r,g,b) if 256 colors suffice: shorter escape sequences (~10 bytes vs ~20 bytes per color).

  • Minimize color changes: group cells by color, or use a limited palette. Ratatui’s diff already handles this somewhat.

  • Use ratatui’s buffer diff (it only sends changed cells). Avoid manually clearing and redrawing the whole screen.

  • Run the simulation at a lower tick rate than the render rate (e.g., 30 ticks/sec simulation, 60 fps render with interpolation).

  • Consider Terminal::insert_before() or partial redraws if only a region changes.

Sources:


6. Half-Block and Braille Rendering Tricks#

Braille Patterns (U+2800-U+28FF)#

Each Braille character encodes a 2x4 grid of dots (8 dots total). The Unicode block contains all 256 possible patterns. The dot positions are:

dot1 dot4    (top row)
dot2 dot5
dot3 dot6
dot7 dot8    (bottom row)

The encoding maps each dot to a bit in the codepoint offset from U+2800:

  • bit 0: dot1 (top-left)

  • bit 1: dot2 (mid-left-upper)

  • bit 2: dot3 (mid-left-lower)

  • bit 3: dot4 (top-right)

  • bit 4: dot5 (mid-right-upper)

  • bit 5: dot6 (mid-right-lower)

  • bit 6: dot7 (bottom-left)

  • bit 7: dot8 (bottom-right)

Resolution: 2x4 = 8 sub-pixels per character cell. Color: Single foreground color for the entire cell. All 8 dots share one color. Background is separate (the cell background color).

Limitation for cellular automata: If you have a sand particle in the top-left dot and a water particle in the bottom-right dot of the same cell, both must be the same color. This makes Braille unsuitable for mixed-material cells unless you sacrifice color semantics.

Half-Block Technique#

Uses three characters:

  • ▀ (U+2580, upper half block): foreground color = top pixel, background color = bottom pixel

  • ▄ (U+2584, lower half block): foreground = bottom, background = top (flipped)

  • █ (U+2588, full block): both halves = foreground color

  • (space): both halves = background color (or just clear the cell)

Resolution: 1x2 = 2 sub-pixels per character cell (vertical split only). Color: Two independent colors per cell (fg + bg). This is the key advantage. Aspect ratio: Terminal cells are ~2:1 (taller than wide), so each half-block “pixel” is approximately square. This makes it the most natural pixel mapping.

Ratatui’s Canvas Markers (v0.30+)#

Ratatui’s Canvas widget supports these markers via Marker enum:

Marker

Resolution

Colors per Cell

Grid Type

Notes

Braille

2x4

1 (fg)

PatternGrid

Default; best resolution, single color

HalfBlock

1x2

2 (fg + bg)

HalfBlockGrid

Best for per-pixel coloring

Quadrant

2x2

1 (fg)

PatternGrid

Dense packing, no visible bands

Sextant

2x3

1 (fg)

PatternGrid

New in 0.30; moderate Unicode support

Octant

2x4

1 (fg)

PatternGrid

New in 0.30; like Braille but densely packed; low Unicode support

Dot

1x1

1 (fg)

-

Simple dot

Block

1x1

1 (fg)

-

Simple block

The colors_rgb Example Technique#

Ratatui’s official colors_rgb example demonstrates the optimal half-block technique for pixel rendering:

// For each terminal cell (x, y):
let fg = colors[yi * 2][xi];       // top sub-pixel color
let bg = colors[yi * 2 + 1][xi];   // bottom sub-pixel color
buf[Position::new(x, y)].set_char('▀').set_fg(fg).set_bg(bg);

This runs at 60fps in the example and renders a full-screen animated color wheel. The technique is directly applicable to cellular automata.

Color Mapping Strategies for Cellular Automata#

  1. Discrete material colors: Each material type maps to a fixed color.

    Empty  -> Black / very dark blue
    Sand   -> Sandy yellow (Rgb(194, 178, 128))
    Water  -> Blue (Rgb(64, 128, 255)), with brightness varying by "pressure" or depth
    Fire   -> Orange-red (Rgb(255, 100, 20)), with flicker variation
    Plant  -> Green (Rgb(34, 139, 34)), with growth-stage variation
    Stone  -> Gray (Rgb(128, 128, 128))
    
  2. Continuous property coloring: For fluids, map pressure/velocity to color hue or brightness.

    fn water_color(pressure: f32) -> Color {
        let b = (128.0 + pressure * 127.0).clamp(0.0, 255.0) as u8;
        Color::Rgb(30, 80, b)
    }
    
  3. ANSI 256 for performance: If 256 colors suffice, use Color::Ansi256(n) for shorter escape sequences.

Sources:


7. Alternative: Offscreen Render to Image, Blit to Terminal#

The Approach#

  1. Render the simulation scene to an offscreen pixel buffer using a real graphics library (e.g., tiny-skia for CPU, wgpu for GPU)

  2. Encode the pixel buffer as a terminal image (Sixel, Kitty, or iTerm2 protocol)

  3. Display the encoded image in the terminal using ratatui-image or viuer

Pros#

  • True pixel resolution: Each terminal cell maps to many actual pixels (font_size.width x font_size.height, typically ~8x16 = 128 sub-pixels per cell). A 200x80 terminal could render at ~1600x1280 actual pixels.

  • Anti-aliasing: Smooth edges, gradients, and sub-pixel detail. Important for fluid simulation visuals.

  • Full 24-bit color at true pixel level: No sub-cell resolution tricks needed.

  • Rich rendering: Can use tiny-skia’s path rendering, gradients, blend modes for sophisticated visuals.

Cons#

  • Terminal protocol dependency: Only works in terminals that support image protocols (Kitty, WezTerm, Ghostty, iTerm2, foot, xterm+sixel). Fallback to half-blocks needed for universal support.

  • Encoding overhead: Sixel/Kitty encoding of a 1600x1280 image at 30fps is expensive. Sixel encoding can take 10-50ms per frame. Kitty protocol is faster but still has overhead.

  • Immediate-mode TUI conflict: Ratatui is immediate-mode (redraw from scratch each frame). Image protocols like Kitty are stateful (load image once, reference by ID). ratatui-image handles this tension but it adds complexity. The StatefulImage widget with ThreadProtocol offloads encoding to a background thread to avoid blocking the UI.

  • Interaction complexity: Mouse/touch coordinates need to be mapped from terminal cells to image pixels. With half-blocks, the mapping is trivial (1:2).

  • No text overlay: You cannot easily render text/UI elements on top of an image in the same area. ratatui-image reserves the image area and skips drawing over it, but you lose the ability to show status text overlaid on the simulation.

  • Larger payload: A full-screen image is hundreds of KB of escape sequences, vs. ~400 KB for half-blocks but only for changed cells. Image protocols typically send the full image each frame (unless using Kitty’s virtual placements).

Verdict for Cellular Automata#

Not recommended as the primary approach. The half-block technique provides sufficient resolution (32K effective pixels), full color semantics, universal compatibility, and better performance. The image protocol approach adds significant complexity and terminal dependency for marginal visual improvement on blocky pixel-art content.

Recommended hybrid: Start with half-blocks. If you later want to add a “high-fidelity mode” for capable terminals, integrate ratatui-image as an optional rendering path with automatic fallback to half-blocks via Picker::guess() or Picker::halfblocks().

Sources:


8. The “Dots Basis” Approach: Efficient Flowing Water Particles#

Concept#

The “dots basis” approach treats flowing water (and other fluids) not as a dense cellular automata grid where every cell must be processed, but as a sparse set of particles/dots that move according to simple physics rules. Only the cells containing particles need to be updated and rendered, which “doesn’t break the bank” computationally.

Why It Works for Terminals#

In a terminal with half-block rendering:

  • Each particle maps to one sub-pixel (one half of a character cell)

  • Only cells that contain particles (or recently changed) need buffer updates

  • Ratatui’s diff-based flushing means unchanged cells cost zero bytes to transmit

  • A 200x80 terminal with 5,000 active water particles only updates ~2,500 cells (since each cell holds 2 sub-pixels), not 16,000

Implementation Architecture#

┌──────────────────────────────────────────┐
│  Simulation Layer                        │
│  ┌─────────────────────────────────────┐ │
│  │  Grid: Vec<Material> (200x160)      │ │
│  │  - Dense grid for solid/stable cells│ │
│  │  - Each cell: 1 byte enum           │ │
│  └─────────────────────────────────────┘ │
│  ┌─────────────────────────────────────┐ │
│  │  Particles: Vec<Particle> (sparse)  │ │
│  │  - Active fluid particles only      │ │
│  │  - {x, y, vx, vy, material, age}   │ │
│  │  - Typically 100-5000 particles     │ │
│  └─────────────────────────────────────┘ │
│  ┌─────────────────────────────────────┐ │
│  │  Update Loop:                       │ │
│  │  1. Update particles (gravity,      │ │
│  │     collision, flow rules)          │ │
│  │  2. Sync particles back to grid     │ │
│  │  3. Render grid + particles to buf  │ │
│  └─────────────────────────────────────┘ │
└──────────────────────────────────────────┘

Particle Update Rules (Falling Sand Style)#

For water particles, simple rules per tick:

fn update_water_particle(p: &mut Particle, grid: &mut Grid) {
    // 1. Try to fall down
    if grid.is_empty(p.x, p.y + 1) {
        grid.clear(p.x, p.y);
        p.y += 1;
        grid.set(p.x, p.y, Material::Water);
        return;
    }
    
    // 2. Try to flow diagonally down (with randomness)
    let dir = if rand::random() { 1 } else { -1 };
    if grid.is_empty(p.x + dir, p.y + 1) {
        grid.clear(p.x, p.y);
        p.x += dir;
        p.y += 1;
        grid.set(p.x, p.y, Material::Water);
        return;
    }
    if grid.is_empty(p.x - dir, p.y + 1) {
        grid.clear(p.x, p.y);
        p.x -= dir;
        p.y += 1;
        grid.set(p.x, p.y, Material::Water);
        return;
    }
    
    // 3. Try to flow horizontally (spreading)
    if grid.is_empty(p.x + dir, p.y) {
        grid.clear(p.x, p.y);
        p.x += dir;
        grid.set(p.x, p.y, Material::Water);
        return;
    }
    // ... more flow rules
}

Growing Plants#

Plants can be modeled as a CA rule with growth propagation:

fn update_plant(grid: &mut Grid, x: usize, y: usize) {
    // Plants grow upward toward light if adjacent to water
    let has_water = grid.is_adjacent(x, y, Material::Water);
    let has_light = y > 0 && grid.is_empty_or_light(x, y - 1);
    
    if has_water && has_light && grid.is_empty(x, y - 1) {
        if rand::random::<f32>() < 0.05 {  // 5% growth chance per tick
            grid.set(x, y - 1, Material::Plant);
        }
    }
}

Performance Characteristics#

Metric

Dense CA (all cells)

Dots Basis (particles)

Cells processed per tick

32,000 (200x160)

~500-5,000 (active particles)

Render updates per frame

Up to 16,000 cells

~250-2,500 cells (changed only)

Bytes flushed per frame (worst case)

~400 KB

~60 KB

Sim tick time

~1-5 ms

~0.1-1 ms

Suitable particle count

N/A

Up to ~10,000 before dense CA is faster

When to Switch from Particles to Dense CA#

The dots basis approach is efficient when the number of active particles is small relative to the grid size. As the simulation fills up (e.g., water filling a pool), the particle count approaches the grid size and the overhead of the sparse representation exceeds a dense update. A practical threshold:

  • < 30% of cells active: Use particle list (sparse)

  • > 30% of cells active: Fall back to dense grid iteration

Rendering the Dots Basis#

impl Widget for &mut SimWidget {
    fn render(self, area: Rect, buf: &mut Buffer) {
        // Clear the simulation area once
        for x in area.left()..area.right() {
            for y in area.top()..area.bottom() {
                buf[Position::new(x, y)].set_char(' ').set_fg(Color::Black).set_bg(Color::Black);
            }
        }
        
        // Render only non-empty cells from the grid
        for (sx, sy, material) in self.grid.iter_non_empty() {
            if sx >= area.width as usize || sy >= (area.height * 2) as usize {
                continue;
            }
            let term_x = area.left() + sx as u16;
            let term_y = area.top() + (sy / 2) as u16;
            let cell = &mut buf[Position::new(term_x, term_y)];
            
            if sy % 2 == 0 {
                // Top half: set foreground
                let fg = material_color(material);
                let existing_bg = cell.bg();
                cell.set_char('▀').set_fg(fg).set_bg(existing_bg);
            } else {
                // Bottom half: set background
                let bg = material_color(material);
                let existing_fg = cell.fg();
                cell.set_char('▀').set_fg(existing_fg).set_bg(bg);
            }
        }
    }
}

Optimizations#

  1. Dirty tracking: Keep a HashSet<(usize, usize)> of cells that changed since last frame. Only iterate those in the render loop. Ratatui’s diff will further reduce the bytes sent.

  2. Spatial hashing: For particle-particle collision, use a spatial hash grid (bucket size = 1 cell) to avoid O(n^2) checks.

  3. Double buffering the grid: Keep two grid buffers. Read from one, write to the other, swap each tick. Avoids in-place update artifacts.

  4. Fixed timestep: Run simulation at a fixed 30 or 60 ticks/sec, interpolate for rendering. Decouples sim speed from render speed.

  5. Material batching: Group cells by material type when rendering to reduce color-change escape sequences. (Ratatui’s diff does some of this automatically.)


Summary and Recommendations#

Decision

Recommendation

Rationale

Rendering approach

Half-block direct buffer manipulation

2 colors per cell, universal compatibility, proven at 60fps by ratatui’s own examples

Framework

Ratatui 0.30+ with crossterm

Mature, active, modular, excellent docs, double-buffer diffing

Resolution

200x160 effective pixels (half-blocks)

Maps 2 sim rows to 1 terminal row; sufficient for pixel-art CA

Color

TrueColor (Color::Rgb)

24-bit color for rich material distinction; use 256-color as optimization if needed

Simulation architecture

Dots basis (sparse particles) for fluids + dense grid for solids

Efficient for flowing water; falls back to dense for filled areas

Image protocol

Not for primary; optional enhancement via ratatui-image

Adds complexity for marginal gain on blocky CA content

Target terminal

Any (half-blocks are universal)

No terminal lock-in; best performance on Kitty/WezTerm/Ghostty

The Project Would Be Novel#

No existing Rust-based terminal falling-sand simulation was found. The Python cellular-automaton-sandbox proves the concept works in terminals. Building this in Rust with ratatui would fill a gap in the ecosystem.

Getting Started#

  1. cargo add ratatui crossterm

  2. Implement a Grid struct with Material enum

  3. Implement a custom Widget that renders the grid to the buffer using half-blocks

  4. Add particle-based water physics

  5. Add plant growth CA rules

  6. Wire up input handling (mouse painting for placing materials)

  7. Profile and optimize (dirty tracking, material batching)