The equalizer

Part 6 of "Building a MIL-STD-188-110 HF Modem in Modern C++." Start with the overview, Part 1, the rules I build by, Part 2, the transmit chain, Part 3, the audio front end, Part 4, acquisition, or Part 5, the first path. The code for this article is on GitHub: kg7hf/modem-core. Part 5 ended with an anchor on the first path, a carrier estimate, the mode and the body boundary. From those, the position of every symbol of the body is known. The symbols still cannot be read. On Table XX's channels each received symbol is the transmitted one plus a copy of a symbol sent 4.8 or 12 symbols earlier, and both copies fade on their own. Read at the anchor, a symbol is two symbols added together, with weights that change several times a second. Undoing that is the equalizer's job. It has help: every frame of data is followed by known symbols, the probes, so the receiver can keep checking its work against symbols it does not have to guess. There are two ways to use them. One is to follow the channel: keep a filter that undoes it, and nudge the filter's taps every time a known symbol shows how far off it is. The other is to measure the channel: estimate from each probe what the channel did to it, and solve for the filter that undoes that. This article builds both, and measures them against each other on the same recordings. Part 5's last paragraph promised a third way, the whole-burst turbo equalizer. It needs the decoder, which is Part 7's subject, so it moves there. Part 5 had first_path_demo; this article has equalizer_demo. It runs on the same recordings, so a build that fetched them for Part 5 has them already: cmake -S . -B build -DMODEM_ARTICLE_DATA=ON cmake --build build ./build/equalizer_demo == Equalizer walkthrough, on the article's recordings == Step Function Section 1 delay_spread() How far behind the anchor 2 window() A window sized from the measurement 3 training() Training on the preamble 4 adaptive() Following the channel 5 cir_mmse() Measuring the channel 6 fade() When the channel moves As before, tests/equalizer_walkthrough_tests.cpp calls the same functions and checks what they returned. Since Part 5 The turbo equalizer moves to Part 7, beside the decoder it runs with. A first clang-tidy cleanup of the modem reached the published files: constants from std::numbers, loop counters as wide as the values they count to, and no mutable global for the error handler. No output changed; the whole-burst audio test checks that. Testing for contributors. One command now builds and runs a named suite: cmake -DSUITE=quick -P tools/test.cmake runs the unit and golden tests, article adds the recordings' tests and the demos (fetching the exact release package if it is missing), and full runs everything the repository registers. A Dockerfile carries the compilers the CI uses, GCC 14 and Clang 19, for anyone who would rather not install them. The recordings' check is itself tested: a damaged, incomplete or padded package must be refused. Every pull request runs the quick suite with both compilers and the article suite in Docker. docs/testing.md has the commands. This article's tests join those suites. How far behind the anchor The equalizer needs to know how long the channel is before it can size itself. Part 5 searched the lags ahead of the anchor for an earlier path. estimate_body_preamble_delay_spread() runs the same kind of search behind it. It correlates the known prefix at lags 0 to 15 symbols, in every preamble segment from the acquired one to the body. It adds the powers across segments, so a path that fades through one segment still counts. Then it subtracts the prefix's own sidelobes; the same idea as Part 5's model, built here from the symbols that follow the prefix instead of the ones before it. What is left is a power for each lag. Lags 0 to 2 belong to the main path: a path that arrives between two symbol times splits its power over neighbouring lags, so those three are never counted as an echo. Echoes cast sidelobes too, so before the scan the estimator removes those as well (more on that below). Then, from lag 3 on, the delay spread is the largest lag whose power reaches an eighth of the strongest (about 9 dB down). If none does, the answer is 0, one clean path. static void cancel_echo_sidelobes(std::array& lag_powers, const std::array& sidelobe_powers, std::size_t maximum_delay_symbols, float significant_power) noexcept { std::array sidelobes_removed{}; for (std::size_t pass = main_path_lobe_lags; pass body_receiver_default_delay_symbols) { return {32U, 6U, 14U}; } return {body_receiver_feedforward_taps, body_receiver_feedforward_taps / 2U, body_receiver_feedback_taps}; } The default holds a 2 ms channel: 24 taps with the reference tap at 12, so 6 symbols ahead of it and 5½ behind, and 8 taps on past decisions. A 2 ms echo is 4.8 symbols late, inside that reach. A spread longer than 6 symbols takes the long shape: 32 taps with the reference tap at 6, so 3 symbols ahead and 12½ behind, and 14 taps on past decisions, enough to reach a 5 ms echo at 12 symbols. Thirty-two feed-forward taps is as large as ProbeDfe goes. The long shape is lopsided on purpose. With the anchor on the first path, the energy that matters arrives after it, so the window spends its taps behind the reference and few ahead. The feedback taps do the rest: an echo of a symbol already decided can be rebuilt from that decision and subtracted. The choice comes from the measurement, never from the mode: a 300 bps burst on a short channel gets the short window. The estimator can only answer 0 or 3 to 15, so in practice 0 to 6 means the default and 7 to 15 the long shape. One rate is different. At 75 bps the receiver keeps the window but sets the feedback taps to zero, because it has nothing reliable to feed back. Feedback cancels the trailing echoes of symbols already decided, one symbol at a time, so it needs a sound decision for each symbol as it passes. At 75 bps each pair of bits is spread over a 32-symbol sequence, and the receiver only knows which sequence was sent after it has correlated all 32. A guess at one symbol on its own is made about 15 dB below the group's signal level, the gain of spreading over 32, and at the signal levels 75 bps exists for it would be wrong more often than right. Fed back, those guesses would add errors instead of removing echoes. So the feed-forward taps alone shape the channel, and the group correlation does the deciding. 2. A window sized from the measurement fp1-echo-group spread 12 -> 32 taps, reference tap 6: 3.0 symbols ahead, 12.5 behind, 14 feedback; trained again on it fp2-window spread 0 -> 24 taps, reference tap 12: 6.0 symbols ahead, 5.5 behind, 8 feedback clean spread 0 -> 24 taps, reference tap 12: 6.0 symbols ahead, 5.5 behind, 8 feedback fp1 gets the long window and is trained a second time on it; the other two keep the default. The demo prints the window's reach the way the receiver uses it: with the reference tap at 6 of 32 half-symbol taps, the window covers 3 symbols ahead of the anchor and 12.5 behind, and the 14 feedback taps reach past the 12-symbol echo. Training on the preamble Once the countdown is read, the whole preamble is known: every symbol the transmitter sent before the body. train_preamble() uses it to train the equalizer before the first data symbol arrives, in three steps. First, a one-tap guess. Over the 288-symbol prefix it measures the channel as a single complex gain, the received symbols against the known ones, and puts that gain on the reference tap. Every other tap starts at zero. Second, practice. It runs the whole known preamble through the equalizer three times, adapting every tap against the true symbol each time, while a carrier loop takes out the frequency error acquisition left. Each pass starts over at the beginning of the preamble, with the carrier loop and the past decisions cleared, but the taps keep what the last pass taught them. The 75 bps short-interleave preamble is shorter, so it gets eight passes. The equalizer's step size depends on the rate: 0.40 for 600 bps and up, which must converge quickly, and 0.10 for 75, 150 and 300 bps, whose 5 ms, 5 Hz channel punishes a step that large with gradient noise. The score is the mean squared error over the last 96 symbols of the last pass. Third, a check on the frequency. Acquisition's frequency estimate is good on a clean channel and can be biased on a fading one. So the score decides. If the first training's error is already at the floor (one percent of the signal power), training is done. If not, the receiver makes up to two more guesses: a fresh estimate from the whole preamble, kept only if it differs from acquisition's by 1.5 to 30 Hz, and a 21-point search across ±30 Hz in 3 Hz steps that fits a channel to the preamble's last 48 symbols at each frequency, where the end points are never allowed to win. Each guess gets a complete training of its own, and the lowest error wins; a tie goes to acquisition. A new estimate never replaces the old one without that measurement, because on a fading channel the estimates themselves fade. Step 3 of the demo shows it: on fp1 the whole-preamble estimate comes out at +21 Hz, where acquisition said +2.55 and the search found -0.51. Taken on trust, it would have trained the equalizer twenty hertz off; scored, it has the worst error of the three and is thrown out. Training is also why the order above is not quite the order the code runs. The spread estimate correlates blocks of 32 symbols coherently, so it needs a good frequency, and the good frequency comes out of training. Both receivers therefore train on the default window first, measure the spread with the frequency that training chose, and train again, from scratch, only if the measurement asks for the long window. A 2 ms burst trains once; a 5 ms burst trains twice. 3. Training on the preamble fp1-echo-group: acquisition's frequency +2.55 Hz its error stayed above the floor, so the receiver trained every distinct guess: acquisition's estimate +2.55 Hz error on the preamble's tail 0.623 a new guess +21.13 Hz error on the preamble's tail 0.741 a new guess -0.51 Hz error on the preamble's tail 0.622 the long window, 32 taps, reference tap 6, 14 feedback; trained again on it training: acquisition's +2.6 Hz stayed above the floor, so the receiver raced 3 guesses and kept -0.5 Hz adaptive: mean probe error 0.682 over 1 block; counting wrong decisions needs the transmitted symbols CIR-MMSE: mean probe error 0.744 over 1 block; counting wrong decisions needs the transmitted symbols == next: soft demapping and the decoder (Part 7) On a message of your own the runner builds the burst itself, so it knows every symbol and counts the decisions: ./build/runner "Hi" --snr 10 --carrier 37 == Part 6 the equalizer: delay spread 0 symbols -> the default window, 24 taps, reference tap 12, 8 feedback training: acquisition's +37.7 Hz stayed above the floor, but no other guess differed from it, so it kept +37.7 Hz adaptive: 0 of 5760 data symbols decided wrong before the decoder (0.0%) CIR-MMSE: 0 of 5760 data symbols decided wrong before the decoder (0.0%) == next: soft demapping and the decoder (Part 7) It still stops before the decoder; that is next week. Go read the code, and the standard In MIL-STD-188-110B 5.3.2.2.2, find the frame of unknown data symbols and known probe symbols for each data rate, and check them against body_block_plan() in body_waveform.hpp. What fraction of the symbols is known at 2400 bps, and at 600 bps? estimate_body_preamble_delay_spread() never counts lags 0 to 2 as an echo. A 1 ms echo arrives 2.4 symbols late: over which lags does its power fall, and what could the estimate report for it? cancel_echo_sidelobes() removes the strongest echo's sidelobes first, not the nearest echo's. What could go wrong if a weaker echo were handled before a stronger one? Work out the long window's reach from 32 taps half a symbol apart with the reference tap at 6. Why does select_body_receiver_geometry() switch above 6 symbols rather than 5, when a 2 ms echo is 4.8 symbols late? The receiver trains on the default window before it measures the delay spread. How long is each coherent block in the estimate, and why does that make acquisition's frequency the wrong one to measure with? inner_trainer_begin() seeds one tap from the prefix and then replays the preamble three times. Why does each replay clear the decision history and the carrier loop but keep the taps? Why does the 75 bps short preamble get eight passes? The frequency race keeps the whole-preamble estimate only if it differs from acquisition's by more than 0.004 and less than 0.08 radians per symbol. What are those limits in hertz at 2400 Bd, and why is there a lower one? In equalize_frame_impl(), a data decision trains the taps only if the output's power is at least 0.25 and its squared distance to the decision is at most 0.5858. Where does 0.5858 come from? Why does a rejected decision still go into the feedback register? The adaptive walk scales each data output by the inverse of the probe error around it. Why does that change no hard decision, and what does it give Part 7's soft decoder? cir_polyphase_solve() works on two problems, one for each half-symbol phase. Using raised_cosine_autocorrelation_half_lag(), show why the noise is uncorrelated from symbol to symbol within one phase. cir_mmse_dfe_solve() adds 2% of the mean diagonal before it solves. What happens to consecutive frames' taps without it, and why does that matter to the interpolation between probes? Run equalizer_compare --seeds 100. Does the 1200 bps row stay close? What does it mean when one engine decides better on most bursts but has nearly the same error rate overall? Next Next week: soft demapping, the decoder, and the turbo loop. The equalizer hands over symbols and its own decisions about them. The decoder works better with less: not a decision, but how sure the receiver is of each bit. Part 7 turns symbols into those probabilities, decodes them, and then feeds the decoder's answers back to the equalizer, which is the turbo equalizer Part 5 promised.