184: Chapter 184 The Cost of Overfitting

October 11th, 6:42 AM.

Room 402, Building 17, Zijing Apartment.

The sky outside was just beginning to brighten, and the thin mist of the autumn morning had not yet dispersed.

Jiang Lin sat in front of his desk, with five overlapping index charts opened side by side on the screen.

[Physical Adjacency Topology]

[Readout Electronics Grouping]

[Measurement Round Expansion Chart]

[Calibration and Reset Events]

[Decoding and Control Strategy Versions]

Two million one hundred thirty-seven thousand eight hundred and sixty-four detection event records converged into a data ocean whose sources were difficult to directly identify.

Now, this sea of data had been re-sliced, categorized, and numbered according to the aforementioned five dimensions.

Among them, forty-two calibration windows had been opened for analysis permissions, allowing them to enter the current processing workflow.

The last window, Window 43, remained sealed.

The USTC team far away in Hefei did not know what method Jiang Lin would adopt, and Jiang Lin could not see any data inside the sealed window.

That was an isolation zone left for the final blind test. Once the analysis protocol was frozen, neither party could modify the rules based on the results anymore.

But before walking toward that door, Jiang Lin had to do something else first.

Reproduce the opponent's original analysis results.

In the logic of scientific review, if a new method could only gain an advantage on paper by misreading the old method, trimming unfavorable data, or secretly changing evaluation criteria, it did not even qualify to enter the subsequent blind test.

Jiang Lin tapped the keyboard and loaded the correlation analysis program submitted by the other party.

The program began searching for patterns within a chaotic expanse.

Following the global chronological order, it accumulated the conditional correlation strengths between different detection events, and then, like panning for gold, filtered out those correlation edges that remained rock-solid across multiple calibration windows.

Pressing enter to run.

The progress bar on the right side of the screen turned into a white line, rapidly advancing to the right.

7:09 AM.

The first global correlation map was generated.

[Full Sweep Results]

Candidate correlation edges passing significance review: 1764

After cross-window stability filtering (stable candidate edges): 186

After rigorous filtering, one hundred and eighty-six ghosts revealed their forms.

Among them, the top twenty-seven edges maintained a correlation strength significantly higher than the background level in more than thirty-six of the forty-two available windows.

According to the opposing team's judgment criteria, these twenty-seven edges were sufficient to clear the suspicion of random noise and were listed as key candidates for physically correlated noise.

The top-ranked edge was highlighted.

[Archive: E-009]

Nodes: D071 — D084

Cross-round offset: +1

Stable windows: 40/42

Average conditional correlation strength: 0.417

Original classification: High-confidence physically correlated candidate

Jiang Lin double-clicked to open its complete record.

The two detection events connected by this edge happened to be located in adjacent physical regions on the chip.

Not only that, in the vast majority of windows, they shared the same set of readout electronics circuitry. In forty of the forty-two calibration windows, this correlation existed stably.

In the charts and reports originally submitted by the other party, E-009 possessed almost all the virtues that a truly physically correlated edge should have.

Stable, strong, and close to the physical hardware topology.

The opposing team even dedicated a full page in the supplementary notes to discuss its significance.

[E-009 may correspond to an unmodeled local correlation error path, such as crosstalk or microwave leakage.]

[It is recommended to add it to the weighted matching graph to reduce decoding mismatch in this region.]

Jiang Lin did not rush to conclusions, but instead cast his gaze toward the five-layer index chart.

Pulling up the physical adjacency graph, E-009 closely adhered to the relative positions on the chip.

Then pulling up the readout grouping graph, in the vast majority of windows, it simultaneously adhered to a fixed readout multiplexing chain.

The physical location and readout structure heavily overlapped here.

Looking solely at these two layers, it was utterly impossible to determine whether it arose from physical crosstalk or circuit multiplexing.

Jiang Lin continued operating, superimposing the third chronological graph.

An anomaly appeared.

E-009 rarely appeared in the same round.

Usually, D071 threw an exception first, and D084 often followed closely in the next round. The average delay was close to a complete measurement cycle.

This was also an important reason why the opposing team regarded it as physical propagation or leaky memory effects.

According to the opponent's original explanation, some physical error might affect the adjacent region after one measurement cycle.

Unmoved, Jiang Lin continued to add the fourth layer.

Calibration, reset, and re-synchronization.

After experiencing these interruption actions, E-009's correlation strength fluctuated slightly, but never completely disappeared. It looked more stubborn and resilient than many ordinary readout faults.

Finally, Jiang Lin superimposed the fifth layer.

Control strategy version.

The seven control strategy versions were sliced by Jiang Lin using a script into seven high-contrast colors, laid out section by section along several months of the experimental timeline.

Right on this layer, E-009's dark red trajectory, which originally spanned almost the entire process and represented high intensity, revealed a break for the first time.

It was extremely weak in the first and second strategy versions, almost non-existent.

Starting from the third version, it rose slightly.

In the first calibration window after the release of the fourth version, the correlation strength suddenly surged, leaping directly from 0.083 to 0.391, and subsequently remained at a high level throughout the entire V4 to V6 period.

When the seventh version went live, within a mere three windows, it quickly plummeted back to the disordered background level.

Jiang Lin dragged his mouse to box and mark the three critical time points.

[Control Strategy V4 Online: E-009 Correlation Strength Surges]

[V4 to V6: E-009 Maintains High Level]

[Control Strategy V7 Online: E-009 Recedes to Background Range]

The timeline fit seamlessly.

This ghost edge, thought to remain stable across calibration windows, did not remain stable across control strategy versions.

Jiang Lin immediately pulled up the version change logs.

Over seven hundred engineering notes were densely arranged by submission time.

Most of them were merely parameter adjustments, log field updates, and boundary condition patches.

He entered instructions to filter for changes related to the physical regions where D071 and D084 were located.

Thirteen remained.

Superimposing another layer of conditions to filter for cross-round control actions: only three remained.

One of them had its submission time frozen on the eve of V4's launch.

[Change Record: PR-1042]

Change Item: High-confidence local correction latency compensation.

Old Strategy: After detection events were confirmed, they entered the queue in the next complete control cycle.

New Strategy: Allow early issuance of local corrections at the end of the current round.

Objective: Reduce the total latency of the decoding and control closed-loop.

Jiang Lin pulled up the underlying control command logs, aligning the timestamps with the events.

The truth began to emerge on the screen like a jigsaw puzzle.

When an exception occurred in D071, the V4 version decoder had a very high probability of selecting a local correction path passing near D084 to reduce latency.

That correction command would be issued through the circuitry to the hardware control queue at the end of the current round.

Just as the next round began, pulses were injected into the chip, affecting the D084 region, and consequently artificially elevating the detection event probability.

The original analysis team only saw the superficial phenomenon: D071 frequently came first, D084 frequently came after, and a stable cross-round correlation existed between the two.

But after incorporating the hidden control logs, the complete causal chain impressively turned into four steps.

[D071 Exception] → [Decoder Selects Local Correction] → [Control Pulse Issued] → [Next Round D084 Event Elevation]

Jiang Lin temporarily masked the control action nodes from the chronological graph, and E-009 turned back into a glaringly strong correlation edge.

Next, he re-added the control action nodes into the model, treating the correction path as a conditional variable, and recalculated the correlation strength between D071 and D084.

The numbers jumped on the screen and then plummeted off a cliff.

[Original Conditional Correlation Strength: 0.417]

[After Adding Control Action Conditions: 0.036]

It dropped directly out of the top five hundred from the very top of the list.

Jiang Lin leaned back against his chair, looking at the new results on the screen, and tapped his index finger lightly on the desk.

This edge indeed existed in reality.

But what it described was by no means how the quantum chip spontaneously generated and propagated physical errors.

What it described was how, after seeing an error, the decoder stepped out a new footprint in the data quagmire of the next round through the control actions it issued itself.

8:03 AM.

Jiang Lin created a new verification record within the system.

[E-009 Attribution Review]

Conclusion: Does not meet physical correlation candidate conditions.

Main reason: The correlation structure appears and disappears with control strategy versions; after conditioning on control actions, the correlation strength recedes to the background range.

Temporary classification: Observation correlation induced by closed-loop control.

[Prohibition: Must not be added to the weighted decoding graph under the name of physically correlated noise.]

After saving, he did not stop, applying the same algorithm and review logic to the remaining one hundred and eighty-five stable candidate edges.

The program no longer blindly sorted according to correlation strength.

Every candidate edge had to first pass five attribution reviews.

Does it adhere to physical adjacency relationships?

Does it migrate following readout groupings?

Does it disappear after calibration or re-synchronization?

Does it change with control strategy versions?

After conditioning on control actions, how much true correlation does it have left?

9:26 AM.

The first round of classification was completed.

The one hundred and eighty-six candidate edges originally mixed in the same stable correlation list were re-divided into four categories.

[Stable Candidate Edge Re-classification List]

Migrates with readout structure: 79

Migrates with control strategy: 48

Source undetermined: 41

Physically correlated candidate: 18

One hundred and eighty-six edges.

Only eighteen remained qualified to proceed into the next round of physical attribution.

And the strongest one among them had been permanently classified into the control strategy-induced zone.

Jiang Lin opened the decoder update scheme submitted by the opposing team once again.

They wrote the top sixty-four stable correlation edges directly into the weighted matching graph, reallocating the cost values of different error paths.

On the existing historical data, the results run by this scheme were quite pretty, and even breathtaking:

[Original Scheme Validation Results (Random Splitting)]

Proxy logic error rate decrease: 9.6%

Average decoding weight decrease: 11.3%

High-error window improvement: 12/14

This was precisely the reason why the other party eagerly hoped to further obtain Jiang Lin's permission to push it toward official deployment.

They knew the independent noise assumption was overly simplistic and crude; after adding these correlation structures, the decoding results indeed visibly improved to the naked eye.

But the problem lay in the validation method.

This validation adopted random splitting.

This meant that data generated under the same control strategy version, such as V4, after being thoroughly stirred, appeared simultaneously in both the training set and the validation set.

This meant that data generated by the same control strategy version, such as V4, after being randomly scattered, entered both the parameter fitting set and the validation set simultaneously.

This correlation weighting scheme did not need to understand the underlying Physics. It only needed to remember the fixed structures repeatedly left behind by a certain control version, such as the compensation pulse corresponding to E-009, to obtain pretty results in identically distributed validation data.

It looked like it had learned to recognize errors.

In reality, it had merely memorized the action habits of the old system.

Jiang Lin wrote a new script to re-divide the data.

The first round adopted leave-one-version-out validation. Each time, a complete control version was left as the test set, while the remaining versions were used to fit weights, checking whether the model could cross general version changes.

The second round used all the first six control versions for modeling, treating the seventh version as the final migration window that never participated in parameter selection.

The two rounds of testing were launched successively.

10:11 AM.

The terminal popped up new test results.

[Model Generalization Capability Test]

[Random Splitting Validation: Proxy Logic Error Rate Decrease 9.6%]

[Leave-One-Version-Out Validation: Average Decrease 0.7%]

[V7 Final Migration Test: Proxy Logic Error Rate Increase 6.4%]

The truth came to light.

Within the familiar version system, this model, which incorporated a large number of pseudo-correlation edges, looked nearly ten percent smarter than the original decoder.

But once the experimental control strategy changed and the old pulse habits disappeared, it was even easier to go down the wrong path than when nothing was added at all.

Because what it remembered was not the stably existing physical laws at the bottom of the chip. What it had memorized by rote were the footprints repeatedly stepped upon day after day by the old system.

10:19 AM.

On the main control screen of the Research Support Unit, the first phase review comments issued by Jiang Lin were received.

The wording was as crisp and straightforward as ever.

[Priority: High]

Please immediately suspend merging the original stable correlation edge set into the formal decoder.

It has been confirmed that at least 48 candidate edges are significantly bound to control strategy versions.

After conditioning on control actions, the correlation of the top-ranked E-009 receded to the background range.

The 9.6% improvement of the original scheme has obvious control version overfitting and cannot be maintained across versions.

In the seventh version independent migration test, the proxy logic error rate substantially worsened by 6.4%.

[Recommended Action]: Arrange a controlled technical review. Participants are limited to the original data team, Professor Lu Zhixing, and personnel designated by the Research Support Unit.

11:07 AM.

The controlled video conference was formally established.

Jiang Lin was still sitting in Room 402, with the background of a monotonous bookshelf and white walls.

On the other end of the screen was a small conference room of the USTC Quantum Laboratory.

Separated by transparent glass behind the wall, massive dilution refrigerators and rows of control cabinets were faintly visible, with indicator lights blinking in the dimness.

Professor Lu Zhixing dialed in from Jiangda University, his video feed located in the lower-left corner, kept on mute.

There were five people in total from the USTC team.

The project leader sat in the center with a solemn expression.

The PhD student in charge of the decoder sat beside him, laptop already open, with dark circles under his eyes from days of staying up late.

Jiang Lin didn't say much else and directly shared his screen.

The first image was E-009, which had consistently ranked first for them over the past four months.

The project leader spoke up first, his voice carrying a hint of defensiveness: "Our team has tracked this edge for a full four months. It crosses most calibration windows and remains closely attached to the hardware's adjacent regions. You're saying now that it has nothing to do with Physics errors?"

"Current evidence is insufficient to support a Physics attribution." Jiang Lin's tone remained unchanged.

He hit enter, overlaying the control policy version color band onto the correlation curves.

Version 4 went live, and the correlation strength surged.

Version 7 went live, and the correlation strength fell back.

The faint murmurs in the conference room instantly vanished, growing so quiet that only the microphone noise floor could be heard.

The PhD student in charge of the decoder changed expression slightly, abruptly turned his laptop to the side, and tapped furiously on the keyboard, searching for version records from months ago.

"V4, V4 did indeed modify the dispatch timing for local calibration," he muttered, his voice somewhat dry.

The project leader frowned deeply, attempting to find a logical loophole: "But this only proves that it coincides precisely with the version update time. Perhaps the state of the quantum system itself happened to change during that time?"

"That's why I performed control action conditioning."

Jiang Lin switched to the second image.

The comparison was stark.

Original correlation strength 0.417.

After control action conditioning, 0.036.

Immediately following that, Jiang Lin presented the comparison results under different paths.

"When D071 appears, but the system does not choose a calibration path passing near D084, the next round of D084 events does not significantly increase. Conversely, when the exact same calibration path is triggered by other completely unrelated detection events, the error rate of D084 still rises."

He paused for a moment to let the other side digest this set of logic.

"This means that in the current data, the main stable correlation between D071 and D084 is not a direct Physics propagation path. The truly stable relationship connects a certain type of control action with the D084 detection event. D071 simply frequently triggers this action, and thus was mistakenly identified as the cause in the raw data."

The PhD student in charge of the decoder stared intently at the conditional stratification chart, looking as if he were stunned.

He slowly pulled out an internal experimental record from four months ago.

The title from back then still remained on the screen — [First Round Confirmation of Stable Correlation Edges].

The first image inside was E-009.

He still remembered that night.

He still remembered that night.

It was the first time they had found this cross-window-maintained strong correlation edge from weeks of continuous experimental records.

The entire group gathered around it, deducing and simulating until three in the morning.

They thought they had finally caught a true Physics noise path.

The selection criteria for the subsequent sixty-four candidate edges were also gradually established using E-009 as a reference.

Now, Jiang Lin simply put the control commands previously treated as result logs back into the causal chain, and that most beautiful edge lost its basis for Physics attribution.

The PhD student fell silent for a full dozen seconds, then spoke with a hoarse voice: "So over these past few months, we've been rising early and working late, just training the decoder to recognize actions we ourselves performed?"

"Part of it, yes." Jiang Lin answered truthfully.

"But the model's performance did improve, that 9.6% isn't fake." The PhD student still harbored a trace of reluctance.

"That's inevitable, because random partitioning put traces of the same control version into both the training set and the validation set at the same time."

Jiang Lin brought up the third table.

Validation Mode Logical Error Rate Proxy Performance Random Partitioning Improvement 9.6% Version Hold-out Improvement 0.7% Cross-Version Migration Degradation 6.4%

The project leader looked back and forth over those three rows of numbers twice, and took a deep breath: "Our team ran the data for Version 7 internally as well. Abnormal degradation fluctuations did indeed occur after adding the correlation edges, but during our team meeting at the time, we thought it was state instability caused by the new round of calibration."

"At the very least, this degradation can no longer be simply attributed to calibration state instability." Jiang Lin interrupted his self-consolation. "After the control policy changed, the correlation structure relied upon by the old model disappeared, yet it still used the old weights to interpret the new system, which is why it went further and further off track."

Professor Lu Zhixing in the lower-left corner timely broke the somewhat solemn atmosphere: "Then do these sixty-four edges all need to be scrapped?"

"No." Jiang Lin switched to the classification results he had spent the morning producing.

Seventy-nine edges, migrating following the readout structure.

Forty-eight edges, migrating following the control policy.

Forty-one edges, undetermined origin.

Eighteen edges, entering Physics correlation candidates.

"Scrapping them directly would throw away valuable information. The correlations produced by the readout structure and the control system are equally part of this experimental system. They reflect problems in the readout, synchronization, or control links. They simply cannot masquerade as Physics noise to participate in decoding."

The project leader asked: "Then how do you plan to handle them?"

"Separate accounting."

Four distinct directories popped up on the screen.

Physical_Correlation_Candidates (Physics candidates, entering the next round of decoding weight review)

Readout_Induced_Correlation (Readout-induced correlation, used to locate circuit and synchronization issues)

Control_Loop_Induced_Correlation (Control-induced correlation, used to check whether the calibration policy is creating new errors)

Unidentifiable_Edges (Unidentifiable portion, remaining unknown to be investigated later)

Jiang Lin paused, his tone leaving no room for argument: "Just because their origins cannot be explained for the time being, we cannot stuff them all into the same Physics noise chart."

The PhD student in charge of the decoder looked at the candidate list with only eighteen edges left and asked: "With only eighteen Physics candidates, is that enough to improve the decoding results?"

"I don't know yet."

"You've already run the internal windows, right?"

"I have."

Jiang Lin cleanly opened up the new results.

Reconstructing the weighted graph using only these eighteen Physics candidates, that dazzling 9.6% improvement instantly shrank to 3.1%.

The expressions of several people in the conference room looked somewhat strange.

3.1%, in this project desperately craving a breakthrough, appeared somewhat meager.

But Jiang Lin opened up the subsequent cross-validation.

Validation Mode Logical Error Rate Proxy Performance Random Partitioning Improvement 3.1% Version Hold-out Improvement 2.8% Cross-Version Migration Improvement 2.6%

It wasn't dazzling enough on familiar data. But after switching to any control version, it didn't suddenly fail either.

Looking at these small yet surprisingly steady numbers, the project leader's gaze changed: "So, although the magnitude of improvement is smaller, the cross-version performance is actually more stable."

"Currently, we can only say that it is more stable. As for whether it actually works, we will ultimately have to look at the sealed window."

Jiang Lin switched to the status page of the independent sealed zone.

[Blind_Window_01]

It still showed as gray.

Prior to this, no one within the system had been able to open it.

The PhD student in charge of the decoder straightened his posture: "When does the blind test start?"

"Go through the procedure first, freeze the analysis protocol, the candidate edge set, and the decoder version."

"Today?"

"Yes, today."

The project leader nodded without hesitation: "The official update of the decoder weights originally scheduled for tonight is suspended. Let's thoroughly investigate the underlying logic of this closed-loop control path first."

He pointed at the red E-009 curve on the screen: "Does this edge still need to be kept in the database?"

"Keep it fully intact," Jiang Lin said.

"It can no longer enter the Physics noise model."

"But it proved a more important thing."

"What?"

Jiang Lin re-expanded the complete causal path of E-009.

[Detection Event]

[Decoding Decision]

[Control Command]

[Next Round Detection Event]

[Correlation Analysis]

[Decoding Weight Update]

If the middle control layer were missing, the entire path would be compressed into a stable, clean, but wrongly-sourced Physics correlation edge.

"It tells us how a decoder turns the traces it leaves behind back into evidence for its next decision."

By the end of the meeting, it was already twelve-forty-eight in the afternoon.

Over at the USTC laboratory, the meeting did not disperse; the entire team turned around on the spot and plunged into a new round of troubleshooting.

The engineers in charge of hardware control began to check the pulse dispatch timings of V4 to V7 frame by frame.

The personnel in charge of data re-extracted and generated the control action index.

The PhD student making the decoder silently deleted the plan to officially update weights tonight, and changed the first item on his task list to — [E-009 Closed-Loop Source Review].

Jiang Lin cut off the connection.

After having lunch and returning to the office at the Mathematics Center, the freezing process officially began.

[Protocol Execution: Syndrome_Identifiability_Test_v0.1]

Analysis Window: 01—42

Sealed Window: 43

Entering Physics Correlation Candidates: 18 edges

Decoder Version: Conservative_Correlation_Decoder_v0.1

Code Hash: f8a (Generated)

Parameter Hash: 3d1ea7f (Generated)

Status: Candidate edge list has been frozen.

All key files separately generated checksums to confirm that subsequent execution packages were not replaced or modified.

Jiang Lin sent the execution package to the Research Support Unit.

After being verified by Professor Lu Zhixing, the Research Support Unit handed it over to the USTC experimental team.

To ensure absolute blind testing, the data of the sealed window would not be transmitted back to Jiang Lin.

The other party would only run the program whose hash value Jiang Lin had already frozen on a physically isolated independent terminal, and would only return pre-agreed result fields without providing any window numbers, event physical locations, or intermediate statistical processes.

At five-thirty-nine in the afternoon, the USTC laboratory sent a definitive reply.

Frozen package hash verification consistent.

Sealed window mounted to the independent terminal.

Three sets of decoding tasks started in parallel.

Estimated runtime: 6 hours.

Jiang Lin refreshed the project status page:

[Syndrome_Correlation_Ledger / BLIND_TEST_RUNNING]

On the right side of the screen, the progress bars representing the three sets of tasks slowly grew from zero simultaneously.

The first was the system's most primitive independent noise baseline.

The second was the aggressive model containing sixty-four strongly correlated edges, which had once brought a massive 9.6% improvement in internal testing.

The third, retaining only eighteen Physics correlation candidates, was the conservative model frozen by Jiang Lin.

In past historical backtests, its performance was the most mediocre.

But at this moment, it was the only set that did not conflate circuit faults, software versions, and unknown phantoms.

Jiang Lin silently watched the beating progress bars for a moment, then closed the remote status page.

After the blind test started, he could no longer modify any parameters.

Otherwise, this window would lose its meaning as an independent verification.

At eleven-twenty-seven at night, the night was already deep.

Outside Room 402 of Zijing Apartment Building 17, everything was dead silent.

In the lower-right corner of the screen, a new message popped up in the fixed communication window of the Research Support Unit.

Sender: Professor Lu Zhixing.

There was only a digitally signed result file.

Jiang Lin straightened his posture, checked the hash code, and double-clicked to open the attachment.

Three lines of cold data clearly appeared in the center of the screen.

[Blind Test Window 43 Results Returned]

Independent Noise Baseline: 1.000

Sixty-Four Edge Correlation Decoder: 1.081

Eighteen Edge Conservative Decoder: 0.963

The results were clear enough.

In the new window that never participated in modeling at all, that sixty-four edge model which had once brought a 9.6% improvement on historical data not only failed to maintain gains, but instead degraded the logical error rate proxy by 8.1%.

The conservative model using only eighteen Physics correlation candidates maintained a 3.7% improvement.

The magnitude was not large, but it passed through the sealed window.

E-009, which originally ranked first, was officially removed from the Physics candidate directory.

It did not disappear from the database, but was placed into a newly named folder.

Control_Loop_Induced_Correlation / E-009

Regarding its project description, there were only two lines of text.

[It was once considered one of the most stable correlated errors inside the quantum chip.]

[It was ultimately confirmed to come primarily from the closed-loop actions of the decoder and the control system itself.]