Pipelining for throughput

The speed of a synchronous design is set by its longest path of logic between two flip-flops. Every signal launched by one clock edge has to settle through the lookup tables on that path and arrive at the next flip-flop before the following edge, so the slowest such path fixes the fastest the clock can run. Pipelining is the main tool for making a design fast. It cuts a long path into shorter pieces by inserting registers, so each piece fits in a shorter clock period, at the cost of a few cycles of latency. On an FPGA, where flip-flops are plentiful and sit beside every lookup table, this trade is almost always worth making.

Latency against throughput

The two words are easy to confuse and the difference is the whole point. Latency is how many cycles pass from an input entering to its result leaving. Throughput is how many results come out per cycle once the pipeline is full. Splitting a computation into stages raises the latency by the number of stages you added, but it leaves throughput at one result per cycle, because every stage works on a different item at the same time. A three-stage multiplier takes three cycles to produce its first product and then produces a new product every cycle after that, forever. For a stream of data, which is what signal processing and inference are, throughput is what matters and the extra latency disappears into the steady flow.

Why registers raise the clock frequency

The clock period has to cover the delay through the logic on the critical path plus the small fixed overheads of the flip-flops themselves. Writing $t_{logic}$ for the logic delay and lumping the clock-to-output and setup times of the flip-flops into $t_{ff}$, the fastest clock is roughly

$$f_{max} \approx \frac{1}{t_{ff} + t_{logic}}.$$

Inserting a register in the middle of that path halves $t_{logic}$ for each half, so each half can run at a shorter period and the clock can speed up, ideally toward twice the original frequency. It never quite doubles, because the fixed $t_{ff}$ overhead is now paid twice and because the logic rarely splits into two exactly equal halves, but the direction is real and reliable. Add stages until the flip-flop overhead, not the logic, dominates the period, and the design is as fast as pipelining alone will make it.

A pipelined multiplier

A wide multiply is a natural thing to pipeline, because the multiply itself is a deep piece of logic. This version registers the inputs, then the product, then the output, giving three stages. A valid bit travels alongside the data through the same registers, so that when a result appears the design knows it corresponds to real input rather than to the pipeline filling with nothing.

pmul.svmodule pmul #(
    parameter int W = 16
)(
    input  logic                   clk,
    input  logic                   rst,
    input  logic                   in_valid,
    input  logic signed [W-1:0]    a,
    input  logic signed [W-1:0]    b,
    output logic signed [2*W-1:0]  p,
    output logic                   out_valid
);
    logic signed [W-1:0]   a_r, b_r;
    logic signed [2*W-1:0] p_r;
    logic v1, v2, v3;
    always_ff @(posedge clk) begin
        if (rst) begin
            v1 <= 0; v2 <= 0; v3 <= 0;
        end else begin
            a_r <= a;   b_r <= b;   v1 <= in_valid;   // stage 1, register inputs
            p_r <= a_r * b_r;       v2 <= v1;         // stage 2, multiply
            p   <= p_r;             v3 <= v2;         // stage 3, register output
        end
    end
    assign out_valid = v3;
endmodule

The valid chain v1, v2, v3 is just a three-deep shift register carrying a single bit, so out_valid is high exactly three cycles after in_valid was, which is the same delay the data takes. The testbench in the repository streams random operands in every cycle and checks each product against a reference queue, confirming both the values and the timing. The same idea, a valid bit shifted alongside data through registered stages, is how the dot product in the capstone keeps track of its results.

Pipelining makes a design capable of a high clock frequency. Whether it actually meets that frequency on real silicon is the job of timing closure, which is the next page.

Next, timing closure.