Automated Tests on Simulation Output, with SQLite

#EDA#VHDL#Verilog#testing#SQLite#bazel#auto

This article demonstrates converting a VCD dump into an indexed SQLite database using go-vcd-parser, then using a query language to answer questions about the signals. A logic simulator answers questions about a digital design. Typically, it answers them in a graphical waveform viewer, requiring an engineer to inspect transitions on a screen. Automating these checks in continuous integration requires translating waveform traces into machine-readable data.

Inside SQLite, engineers write Go tests to assert timing properties, such as validating that reset deasserts at 10 ns or verifying a 200 MHz clock. These checks execute automatically and fail the build whenever an assertion violates.

Why not assert inside the simulation

The conventional way to test a design is by placing assertions directly inside the simulation. Developers use VHDL assert statements, SystemVerilog Assertions (SVA), or verification libraries like OSVVM and UVVM. These remain the right tools for evaluating cycle-by-cycle behavioral properties during simulation execution.

However, post-simulation queries are often simpler to evaluate after execution completes. Broad questions (such as scanning whether any hierarchical signal remained unresolved after reset) require global sweeps over recorded history. Engineers frequently identify these queries only after diagnosing simulation anomalies. Furthermore, waveform dumps received from external vendors or bug reports rarely include internal assertions.

A VCD file is a chronological record of value changes paired with timestamps. This structure forms a natural database table. Once loaded into a database, waveform evaluations become reproducible tests.

From a dump to a database

Converting a waveform requires a single command:

# from a simulation, e.g. nvc -r --wave=dump.vcd tb
vcdcvt -in dump.vcd -format sqlite -out signals.db

On an 11 MB dump containing 1.2 million lines, conversion takes roughly two seconds and uses approximately 19 MB of memory. The parser streams records, and the database loader batches inserts while deferring index construction until completion. Memory usage scales with the number of unique signals rather than simulation duration, allowing long simulation runs to process on standard workstations.

The generated database contains three tables. The dbq Go package provides a query layer that abstracts database operations into signal lookups and timestamps.

Writing a unit test

Testing query logic does not require executing a full simulator. The dbt package populates an in-memory database using explicit time and value pairs, following the test patterns found in dbq/pkg_test.go. This setup produces fast, hermetic unit tests for checking logic.

dbx, _ := db.OpenDB(ctx, dbt.NewMemDB())
i := dbt.New(dbx, ctx)
i.Signal("//clk", vcd.VarKindLogic, 1).
    //
    // //clk   ________/~~~~~~~~~~...
    //         ^0      ^100
    TimeValues(dbt.TimeValue{Time: 0, Value: "0"},
        dbt.TimeValue{Time: 100, Value: "Z"})

q := dbq.New(dbx)
ts := q.Signal("//clk").FindFirst("Z")   // ts.T() == 100

The package provides composable query primitives: FindFirst(value), FindAfter(ts, value), FindBefore(ts, value), EqAt(ts, value), PrevChange(ts), and NextChange(ts). Two lookups differ in a critical timing detail: ValueAt(ts) returns the signal value immediately before a transition at ts, while ValueAtP(ts) includes the transition occurring at ts. Confusing these methods introduces off-by-one edge errors.

Every lookup returns a Timestamp object that can report an error:

ts := clk.FindFirst("1")
if err := ts.IsOk(); err != nil {
    t.Fatalf("clock never rises: %v", err)
}

The dbq.FindFirst helper combines multiple predicates to identify the first instant when all conditions hold simultaneously:

r := dbq.FindFirst(
    func(ts *dbq.Timestamp) *dbq.Timestamp { return s1.FindAfter(ts, "1") },
    func(ts *dbq.Timestamp) *dbq.Timestamp { return s2.EqAt(ts, "2") },
    func(ts *dbq.Timestamp) *dbq.Timestamp { return s3.EqAt(ts, "3") },
)

Testing against simulation artifacts

These queries run identically against databases generated from simulator dumps. Under Bazel, the conversion executes as a standard build action, producing a cacheable database artifact:

load("//:macros.bzl", "vcd_index", "vcd_go_test")

vcd_index(
    name = "tb_example",
    vcd_target = "tb.vcd",
)

vcd_go_test(
    name = "dbq_test",
    srcs = ["pkg_test.go"],
    embed = [":dbq"],
    vcd_file = "//vcd/files/samples:tb_example",
)

The vcd_index macro invokes vcdcvt inside a genrule. The vcd_go_test rule wraps go_test, passing the database file as runfiles data through --test-db-name. Inside the test, dbq.GetTestDB() accesses the database:

func TestResetReleasesBeforeTraffic(t *testing.T) {
    dbx, _, err := dbq.GetTestDB()
    if err != nil {
        t.Fatalf("no test database: %v", err)
    }
    q := dbq.New(dbx)

    release := q.Signal("//wb_uart_tb/reset").FindFirst("0")
    if err := release.IsOk(); err != nil {
        t.Fatalf("reset never released: %v", err)
    }
    if got, want := release.D(), 10*time.Nanosecond; got != want {
        t.Errorf("reset released at %v, want %v", got, want)
    }
    // reset_n is the inverse, and must already be high here.
    resetN := q.Signal("//wb_uart_tb/clkgen/reset_n")
    if v := resetN.ValueAtP(release); v.V() != "1" {
        t.Errorf("reset_n is %q when reset releases, want \"1\"", v.V())
    }
}

The test operates on build outputs inside the Bazel sandbox without managing simulator execution. Tests run in parallel, and assertion errors identify specific signal paths.

Signal names follow full hierarchical paths, such as //wb_uart_tb/clkgen/reset_n. Executing vcdcvt with -signals signals.csv generates a CSV summary listing every signal name, data type, and bus width.

Timing assertions and timescale units

The Timestamp.D() method returns a standard Go time.Duration. Three helper functions build upon this representation: Diff(a, b) measures the interval between two timestamps, IsDurationApprox(a, b, d) asserts that two transitions occur within interval d, and IsClock(from, sig, freq) validates rising edges, falling edges, and half-period symmetries:

clk := q.Signal("//wb_uart_tb/clk")
if err := dbq.IsClock(&dbq.TimestampZero, clk, 200e6); err != nil {
    t.Errorf("clk is not a 200 MHz clock: %v", err)
}

A raw VCD timestamp represents an integer count of timescale units declared in the file header via $timescale. Simulators use different default precisions: nvc often outputs 1fs, whereas Vivado xsim defaults to 1ps. A counter value of 1,000,000 corresponds to one nanosecond at femtosecond resolution, but one microsecond at picosecond resolution.

The database records the active timescale, allowing dbq helpers to scale counts into accurate durations. Go’s time.Duration tracks whole nanoseconds. Sub-nanosecond skew in a femtosecond dump rounds to zero under D(). For sub-nanosecond measurements, use Fs() to obtain exact femtosecond counts:

skew := later.Fs() - sooner.Fs()   // femtoseconds, exact

Visualizing test failures

When an assertion fails, the test reports the affected signal identifier and timestamp. A companion utility extracts the relevant simulation window into a timing diagram:

sqlite2drawtiming -in signals.db -min-time 0 -max-time 40000000 \
    -ndots 2500000 \
    -signal '//wb_uart_tb/clk=>clk' \
    -signal '//wb_uart_tb/reset=>reset' > timing.dt
drawtiming -o timing.png timing.dt

This workflow enables automated regression checks in CI, rendering graphical traces only when failures require inspection.

Querying with raw SQL

Because the output is a standalone SQLite database, developers can execute raw SQL queries directly. SQL is useful for sweeping across all design signals or inspecting global simulation runs. The schema defines three tables:

Signals(Name, Type, Code, Size)     -- Type 17 is logic, 18 is string
Svalues(Id, Timestamp, Code, Value) -- Code joins to Signals.Code
Meta(Key, Value)                    -- generator, timescale, timescale_seconds

A single query can scan for unresolved signals following reset deassertion:

WITH settled AS (
  SELECT s.Name AS name, v.Value AS value,
         ROW_NUMBER() OVER (
           PARTITION BY v.Code ORDER BY v.Timestamp DESC) AS rn
  FROM Svalues v JOIN Signals s ON s.Code = v.Code
  WHERE v.Timestamp <= 20000000   -- 20 ns, after reset releases
    AND s.Type <> 18)             -- 18 is $var string
SELECT name, value FROM settled
WHERE rn = 1 AND value GLOB '*[XxUuZzWw-]*'
ORDER BY name;

The ROW_NUMBER() ... PARTITION BY v.Code window function identifies the final logic state recorded before time T. On a testbench with floating nets, this query flags undriven signals directly.

Note two SQL query details: SQLite’s LIKE operator is case-insensitive for ASCII characters, so value LIKE '%X%' matches letters in identifiers like rx_get_start_bit. The GLOB operator is case-sensitive, distinguishing signal values from identifier strings. Because scalar values might appear as lowercase u while vectors record uppercase X, the character class pattern GLOB '*[XxUuZzWw-]*' matches all unresolved states.

Timestamps in SQL represent raw simulator ticks. Multiplying by timescale_seconds from the Meta table converts ticks to seconds. Lookups remain efficient because the database maintains an index over Svalues(Code, Timestamp, Value).

Implementation notes

  • The database exists as a single standalone file, allowing Bazel to cache and distribute it like any build output.
  • Multiple waveform converters (including .wdb tools) can target the same schema, enabling uniform verification suites across toolchains.
  • Post-simulation database queries complement in-simulation assertions by providing global analysis across entire test runs.