82: Chapter 82 Reproduction Package

Jiang Lin closed the audit_log file that bore the words [Next Execution Target] and created a blank Python file.

minimal_leak_demo.py

With the 24-hour window demanded by the opposing party, twenty-three hours and forty-one minutes still remained.

Time was sufficient, but he had no intention of slackening off in the slightest.

Constructing a minimal reproduction experiment sounded simple, but in practice, it was a task that required a delicate sense of proportion.

First, not a single byte of the 3.4 GB raw dataset provided by the platform could be used.

Otherwise, the other party could turn around and bite back, claiming that he was taking things out of context by exploiting some special structure inherent to the dataset itself.

Second, it could not be overly complex.

If the script was written in a flashy manner with a bunch of weird libraries depended upon, the opposing engineers would waste half a day just configuring the environment during the review. Once the review process dragged on, the matter would easily end up being cold-treated.

Third, the random seed must be completely locked down.

Any minute fluctuation in a floating-point number could be attributed by the opposing technical team to environmental differences, thereby slowing down the review process.

Jiang Lin stared at the blank code window, thought for two minutes, and then began working, using numpy to generate a random time series spliced together from two different distributions.

The first eighty data points followed a normal distribution with a mean of zero and a standard deviation of one, used to simulate the stable market conditions within the training window.

The last twenty data points had several extreme values exceeding three standard deviations artificially injected into them by him, used to simulate abnormal events suddenly appearing in the testing window.

The entire data construction process took no more than ten lines of code.

The random seed was locked to 42.

To avoid version discrepancies in the random number generator, he simply deleted the random generation process and directly hard-coded a set of minimal synthetic sequences into the script.

The numbers carried no financial meaning whatsoever and were only used to prove a mathematical fact.

Participation of the testing window in standardization would systematically depress the anomaly scores of the extreme values within it.

After the data was ready, he wrote two control functions below.

The first function was called leaked_scaler, which completely copied the logic of line 67 of the Baseline.

It merged the training window and testing window, calculated the global mean and global standard deviation, and then performed standardization.

The second function was called causal_scaler, strictly adhering to temporal causality.

It used only the mean and standard deviation of the training window to standardize the testing window.

Then, he had the two functions process the same piece of synthetic data respectively and printed out the anomaly scores after standardization.

Finally, he added a line of matplotlib plotting code to overlay the two sets of results onto the same graph.

The total length of the code was less than forty lines.

Jiang Lin clicked run.

In less than a second, that control graph popped up on the screen.

It was almost identical to the results previously run with real data.

The anomaly score curve processed by leaked_scaler was nearly a section lower than that of causal_scaler.

Those spikes that should have pierced the sky were quietly pressed down by future information.

Clean, pure, and irrefutable.

Jiang Lin captured a screenshot of this control graph and saved it into the folder.

Next, he did something even more ruthless.

He wrote a README explanatory document for this minimal reproduction experiment that looked like a laboratory equipment calibration record.

The first section of the document was the statement of purpose for the script.

"This script aims to use purely synthetic data to reproduce the future information leakage phenomenon in the preprocessing stage observed in the real dataset. This script does not rely on any raw data provided by the organizer at all, and all random numbers used are generated by a locked seed of 42, which can theoretically yield completely identical output in any Python environment supporting numpy version 1.24 or above."

The second section of the document contained line-by-line code comments.

The third section of the document provided the exact numerical values of the expected output.

Jiang Lin printed both sets of anomaly score sequences and the specific numerical values of every time point into the README in a tabular format.

Anyone who ran the script once could use the diff command to compare byte by byte whether their output matched his output.

The fourth section of the document was mathematical derivation.

Spanning three pages, starting from the most basic standardization formula, he stepped-by-step derived why the inclusion of testing set samples in the calculation would lead to the systematic underestimation of extreme values within the testing window after standardization.

Every derivation step was numbered, and every formula had a citation source.

The fifth section of the document was engineering recommendations.

He calmly listed three correct preprocessing schemes and marked their respective applicable scenarios and computational overhead.

The final section was the disclaimer.

"All code, data, and derivation processes of this experiment do not involve any internal information of the desensitized dataset provided by the organizer. This experiment aims to independently verify a mathematical fact in a minimized manner, and has nothing to do with the specific implementation details of the organizer's platform."

Once this README was sent out, the other party would completely lose all room for ambiguous handling.

He used synthetic data, so the other party had no way to say he was maliciously attacking the platform.

He locked the seed and attached exact numerical values, so the other party had no way to say it was unreproducible.

He provided three pages of mathematical derivation, so the other party had no way to say he lacked theoretical backing.

What he gave were suggestions, not accusations, so the other party had no way to say he was bossing people around.

This was like building a minimally closed-loop control apparatus in a Physics experiment.

The inputs were fixed, the processing flow was fixed, and the output difference was fixed.

You might not like the conclusion, but you could not say it did not happen.

Jiang Lin glanced at the time in the lower right corner and packaged the entire folder into a file named reproduce_leakage_v1.0.zip.

Then he switched back to the browser, opened that internal message, and clicked the supplementary materials upload button.

The page prompted: Supplementary materials have been submitted, awaiting review by the review panel.

Jiang Lin closed the browser, stood up from his chair, walked over to the window, and pulled open a slit in the curtains.

The sunlight outside was so dazzling that it made him squint his eyes.

Only then did he realize that he had stayed inside the room consecutively for nearly five days.

He turned around and went to take a bath.

The moment the hot water washed over the back of his neck, the string in his mind that had been tightly wound for four days finally loosened just a bit.

While drying his hair with a towel, he heard his mother speaking in a low voice with his father in the living room.

"He has been shut up in the room for five days, and even his meals were carried in by me."

"It is a good thing for young people to be busy with matters, don't go disturb him."

"I am just worried."

"The kid can eat and drink fine, and his condition is great. Stop worrying blindly."

Jiang Lin's motion of wiping his hair paused for a moment.

He thought about it, draped the towel over his neck, pushed open the door, and walked out.

"Mom, I want to go out for a walk this afternoon."

His mother turned her head from the sofa to look at him, smiles instantly piling up on her face: "Oh, go ahead then, Mom will transfer you some pocket money."

...

Meanwhile, Shanghai, on the 23rd floor of an office building in Pudong.

Six people were sitting in the internal meeting room of the platform's technical team, and no one spoke loudly.

The more low-pitched the voices were, the weightier the matter actually was.

Sitting at the head of the long conference table was a man in his forties wearing gold-rimmed glasses, who was the technical lead for this challenge and had the surname Shen.

On his laptop screen, page 18 of a PDF was currently displayed.

Two lines of small text were printed at the header position.

"《Minute-Level Market Data Anomaly Detection and Backtesting Pre-Audit Report》"

"Submitter: No. 1453."

Engineer Shen's right index finger unconsciously tapped on the table, the rhythm getting faster and faster.

"Have we found out the background of this No. 1453?"

A young engineer wearing headphones sitting on his left immediately shook his head: "The registration information only has a mobile phone number and an email address. We can only see the region of the mobile number and the email domain proactively left during registration. Deeper identity verification cannot be done; the user agreement traps us very tightly."

"What do you guys say, what kind of identity could the other party have, a graduate student from some university?" Engineer Shen frowned.

"Unlikely." Another slightly older female engineer shook her head. "Graduate students are accustomed to using LaTeX when writing reports, and their phrasing also has the flavor of academic clichés. But the writing style of this report is very..."

She hesitated for a moment, finding an appropriate adjective.

"Very industrialized."

"Engineering-oriented?"

"It is the kind of tone that can only be written by someone who has done hands-on work on the front lines. Look at the pollution link topology map he drew on the second page; he is not looking at this data from a financial perspective at all, but rather from the perspective of a measurement system."

"He probably doesn't care about the market at all, and only cares about every layer of data transmission between the exchange and the platform, and which layer is most likely to drop packets, drift, or misalign."

The female engineer turned to page 8 of the report and turned the screen towards Engineer Shen.

"The real trouble is not that he pointed out horizontal missingness. Horizontal missingness can be seen by anyone who is careful just by drawing a heat map. The trouble is that he put these missing bands, price scale jumps, and source_flag group changes into a single graph."

Engineer Shen lowered his head to look at the screen.

On page 8 of the report, there was a three-layer aligned chart.

At the very top was the missingness heat map expanded by minute_id, and in the middle were the price scale changes of several desensitized targets at the boundaries of adjacent date_ids.

At the very bottom was a discretized source_flag grouping sequence.

Between the three charts, Jiang Lin drew out several sets of correspondence relationships with thin gray lines.

Below the chart, he wrote only a very cold judgment.

[The true business meaning of source_flag is unknown, and this article does not make a definitive explanation for it. However, the grouping changes of this field have a statistical correlation with several horizontal missing bands and scale mutation boundaries. It is recommended that the organizer hierarchically recheck the underlying data sources, cleaning batches, or supplier interface switching records according to source_flag.]

The meeting room fell silent for a few seconds.

The young engineer said in a low voice: "He didn't say that source_flag is the supplier ID."

The female engineer nodded: "Right, he didn't overstep, he only proved that this field is not a harmless field and is related to the anomaly distribution."

Engineer Shen's expression sank.

Because they themselves knew that behind source_flag was indeed not an ordinary quality tag.

What it corresponded to was a mixed marker after two underlying market data suppliers and an internal cleaning batch switch.

This field was supposed not to be exposed so completely.

What was even worse was that No. 1453 didn't guess it by relying on the field name.

He reverse-engineered this link out from data faults, scale jumps, and grouping synchronous changes.

"In other words, he doesn't know who our underlying supplier is, but he already knows that a data source switch occurred here," Engineer Shen mused.

The female engineer said: "Yes, and he is very smart; he didn't push his words to absolute terms in the report, only suggesting that we perform hierarchical rechecks."

This was harder to deal with than direct accusations.

The other party neither revealed any emotion nor gave any exaggerated conclusion, and didn't even say a single overstepping word.

They simply placed the evidence there, and then wrote down clearly for them what should be checked next.

Engineer Shen thought for nearly a minute, then raised his head to look at the legal counsel across the table who had not spoken all along.

"Old Zhou, among the top ten on this public leaderboard, at least several high-scoring submission descriptions clearly followed the global standardization process of the baseline. If his Class E section were really posted to the external internet?"

The middle-aged man called Old Zhou leisurely pushed up his glasses.

"If he has no intention of malicious public disclosure, then it is a very rigorous internal audit recommendation, and we should thank him. But if he chooses to post this report on Zhihu, Xueqiu, or any quantitative forum, it will be a serious industry-level accident."

"The problem is not the baseline itself," the female engineer said. "The problem is that the data cleaning templates we demonstrated to clients in the past few batches were also modified from this set of notebooks. Contestants can choose not to use them, but clients will look at them. As long as he sends out this control experiment, the outside world will not distinguish between sample code and platform data capabilities."

Speaking up to this point, Old Zhou glanced at Engineer Shen and explained: "Therefore, all public leaderboard scores of this batch of our challenge will lose credibility from both legal and engineering perspectives. Because previous batches of the challenge also used a similar Baseline, they will be questioned together. Several institutional clients who have already signed commercial cooperation will immediately issue letters demanding us to conduct technical audits."

"The worst-case scenario is that the reputation and business of our entire platform technical team within the next three years will suffer irreversible damage."

The meeting room once again fell into a lifeless atmosphere.

Just at this moment, a new email notification suddenly popped up on the computer screen in front of the young engineer wearing headphones.

He clicked to open it and take a look, his pupils shrinking abruptly.

"Engineer Shen, No. 1453 just submitted supplementary materials called reproduce_leakage_v1.0.zip."

Engineer Shen's breathing paused for an instant, and he immediately said: "Open it and project it onto the big screen."

The young engineer scanned the archive of less than 100 KB, then downloaded and extracted it, projecting the contents of the README onto the large screen in the meeting room.

Six people held their breaths simultaneously, their gazes sweeping across the screen from top to bottom.

For a full two minutes, no one spoke.

Finally, that female engineer let out a soft breath: "He didn't even leave us any room to look for excuses."

Engineer Shen stared at the expected output table on the screen accurate to sixteen decimal places, slowly leaning back against the back of his chair.

After a long while, he finally spoke: "Notify the review panel, temporarily pause the automatic publishing of the final scores, and wait for the technical review results. Also, contact No. 1453, I want to talk to him."

...

When Jiang Lin returned from outside, it was already past six o'clock in the evening.

At the dinner table, his mother rarely nagged him about staying cooped up in his room these past few days, and instead kept piling food into his bowl.

After finishing dinner, he returned to his room and habitually shook the mouse.

The screen lit up.

The notification bell in the upper right corner of the competition webpage displayed another new internal message.

"Preliminary Communication Letter Regarding Inviting You to Participate in the Platform Data Audit Special Project."

The sender still carried a blue V-certification: Platform Technical Team Administrator - DataOps_03.