84: Chapter 84 First Order
At 7:56 PM, Jiang Lin was still staring at the budget spreadsheet on the screen.
The spreadsheet's structure had expanded even further compared to a few days ago.
[Specialized Books and Literature Database Subscriptions]
[Basic Computing Equipment and Storage]
[Electronic Measurement and Signal Analysis Equipment]
[Machining Tools and Basic Consumables]
[Data Services and Commercial Software Licenses]
[Advanced Instruments That May Be Needed in the Future But Whose Purchase Is Postponed]
When each item was expanded, densely packed rows of prices, uses, priorities, alternative solutions, and risk notes trailed underneath.
A small number of entries were highlighted in green, meaning they had to be acquired immediately no matter the cost.
Several yellow items meant waiting for the first batch of equipment to be up and running and purchasing them later when there was spare money.
But scrolling down, large swathes of depressing gray on the screen silently reminded him of one fact.
The money was not enough.
At 7:59 PM, Jiang Lin pressed Ctrl + S to save the spreadsheet and switched back to the competition homepage.
At 8:00 sharp, as the spinning circle in the top right corner of the browser stopped, the homepage banner refreshed on time.
[Closing Announcement for the Third Phase of the Minute-Level Market Data Anomaly Detection Challenge]
The first paragraph of the announcement was dry bureaucratic jargon.
Thanking the contestants for submitting algorithms, reports, and reproduction materials, and thanking the technical support units.
But starting from the second paragraph, the tone changed drastically.
Reviewed by the technical team, the official Baseline for this installment had an issue with mixing training window and testing window statistics during the feature preprocessing stage, which might have affected some submission results that continued to use this workflow.
The platform has adjusted the final evaluation rules accordingly; automatic scoring is no longer the sole ranking basis, and final results will comprehensively consider hidden test set performance, report logic, reproducibility, generalization ability, and time series causal isolation principles.
Mixed statistics!
Jiang Lin looked at these five words and the corner of his mouth twitched.
The platform's wording was really restrained.
It didn't mention future information leakage, serious accidents, or official code errors.
But anyone who had genuinely worked with data on the front lines understood everything without needing it spelled out.
In time series tasks, having the training set and testing set share statistics was essentially stuffing the future into the past in the Physics sense.
The entire competition leaderboard had essentially turned into a contest of who could use tomorrow's answers to take today's exam more seamlessly.
The announcement continued.
This technical review material was submitted by Contestant No. 1453.
In the methodology report, this contestant systematically analyzed the data contamination pipeline, missingness mechanisms, field consistency, and official Baseline preprocessing issues, providing an independently runnable minimal reproduction experiment.
After discussion by the evaluation panel, Contestant No. 1453 won the Special Technical Contribution Award for this challenge.
...
The mouse wheel scrolled to the bottom, and the final rankings were released.
First place was still first place, and those with high automatic model scores basically retained their rankings.
The platform still needed to maintain the decency of the competition and the continuity of the rules.
But next to the original leaderboard, a conspicuously abrupt column of tags had suddenly appeared.
No. 1453's automatic score ranked seventh.
Manual Review Grade: S.
Final Comprehensive Ranking: Third.
Special Technical Contribution Award: Listed separately.
This result was exquisitely subtle.
The platform hadn't crudely pushed him up to first place, but neither had it hidden away the report that exposed the stark truth.
That glaring column of S grades in the public evaluation system usually meant it completely exceeded the expected evaluation scope.
The platform was using an extremely restrained—and even slightly aggrieved—manner to announce to everyone in the industry who the truly hardcore player of this competition was.
Jiang Lin casually clicked open the discussion area.
In just a few minutes, the first batch of posts had sprung up like bamboo shoots after a rain, and the entire forum section was in an uproar.
[What does Baseline leakage mean? Begging the experts to explain in human language!]
[F*ck, I knew it! I modified it based on the official notebook, why was there a world of difference between offline and online!]
[Are the automatic top five scores still valid?]
[Who is 1453? Can the report be made public?]
[Anomaly detection competition looks at reports instead of F1 score, is this reasonable?]
[The official dug a hole themselves and made the contestants take the blame? Have they no shame changing the rules at the last minute?]
The person who posted and cursed the official for changing the rules temporarily was immediately cornered by the replies below.
"If your solution didn't copy the official Baseline, what are you panicking about? Apply for a manual review from the platform."
The original poster said nothing more, and a few minutes later, the main post was sneakily deleted.
A string of mocking sneers was left in the discussion area.
Some people had even sharper words.
"The problem now is, how do those people on the leaderboard prove their innocence? I've read several top-ten submission descriptions, and they don't even f*cking write how normalization is done, but their model names are piled up longer than top conference papers."
This sentence was like a precise slap to the faces of certain users who didn't dare to show their faces.
As time passed, the topic inevitably focused on Jiang Lin's account.
[Seventh place gets a Special Contribution Award and is even given an S? What is this, a dimensional reduction strike?]
[Looking at the announcement description, it's not that his model score is high, but that he directly dismantled the competition data pipeline.]
[This type of contestant is the most annoying, forcefully turning a competition match into a code audit report.]
[Annoying as it is, in all honesty, if it weren't for this guy, this leaderboard might have actually been published as a scientific research achievement. The top five were all carrying water together, who would voluntarily admit it?]
[Report! I just flipped through No. 1453's public submission summary. The methodology description is over thirty pages long, and the appendix even draws a missing mechanism classification tree. This is not a contest report, this is a data compliance white paper. If the evaluation group gives a low score, they'd probably find it too hot to handle.]
[No wonder the manual review gave an S. Reviewing this kind of thing, the evaluation panel would be embarrassed to give a low score.]
...
Loving to watch the chaos, the top-ranking contestants finally couldn't sit still.
First place's account posted a long statement with a steady tone.
He emphasized that his solution did not use the official global normalization process, was willing to accept code review, and finally added a decent sentence: "Thanks to Contestant No. 1453 for their hard work, this is meaningful for the scientific nature of the entire competition."
A clever calculation.
Taking the initiative to clarify and praise, pulling himself out of the quagmire of leakage suspicion as quickly as possible.
But second place clearly didn't have such good composure, and his words were tinged with anger.
He emphasized that deep models have a natural advantage for complex time-series anomalies, and model methods should not be denied because of a single report.
[Deep models have a natural fitting advantage for complex time-series anomalies. Methodological rigor shouldn't override practical results; competitions are judged by results, not written as academic review papers.]
Less than two minutes after this post was published, someone questioned him below.
[No one is denying deep models. The question is, what did your proud model actually learn—real anomalies, or the contaminated global distribution?]
Second place didn't reply.
After a while, another user added a follow-up blow.
[OP, stop being stubborn. I just downloaded your reproduction code. Line 87, sklearn's StandardScaler, you directly ran a fit_transform on the entire dataset. This is not called prediction, this is called an open-book exam.]
The comment section instantly blew up.
Screenshots of the code and score comparison charts before and after the modification flooded in like snowflakes.
Someone isolated the training and test sets and re-normalized them, and the resulting score plummeted by nearly fifteen percentage points.
[Getting the statistical distribution of the answers before the exam and then claiming you didn't cheat doesn't really fly, does it?]
[It wasn't cheating; it was being scammed. More than half of the contestants in the competition were using this preprocessing pipeline.]
[Being scammed is a fact too; admitting it is fine, nothing shameful about it. It's just that some people are still refusing to admit defeat, which looks a bit ugly.]
The public sentiment completely reversed within twenty minutes.
Second place never replied again.
Seeing this, Jiang Lin closed the discussion area.
The truly expensive reproducible materials, he had already handed over to the platform, and the platform had also written the most crucial facts into the announcement.
The remaining arguments were outside his scope of concern.
He saved the announcement page as a PDF, and organized the email sent by Shen Chengye, the draft agreement, and the final confirmation email into a folder.
The folder name was still: [Technical Service Boundary]
Then, he opened the budget spreadsheet.
Under the [Independent Funding Source] sheet, that original row's status could now be updated.
Project: Challenge Special Technical Contribution Fee
Amount: 100,000 RMB
Status: Agreement confirmed, waiting for platform payment
One hundred thousand.
For an ordinary family, this was not a small sum of money.
But placed on a scientific research equipment list, it was as thin as a piece of paper.
He switched to the [Basic Computing Equipment and Storage] sheet.
The first item originally marked in yellow was changed by him to green.
Second-hand dual-processor workstation, with at least 256GB of memory.
Four enterprise-grade hard drives.
UPS.
Added together, these items had already consumed the majority of the budget.
He switched again to [Electronic Measurement and Signal Analysis Equipment].
USB logic analyzer, green.
Portable oscilloscope, green.
Multi-channel voltage and current recorder, green.
Low ESR capacitors, ceramic capacitors, ferrite magnetic rings, and a batch of basic consumables, green.
Temperature and humidity recorder, green.
Tri-axial acceleration sensor, yellow.
Small high-sampling-rate data acquisition module, yellow.
Current probes, high-voltage isolated probes, NI data acquisition cards, still gray.
The 100,000 yuan failed to dispel all the gray; it merely lit up a path in the dim mist just wide enough for him to walk out.
Jiang Lin was not disappointed.
No system is built on a single payment.
The significance of this money was not to provide him with complete experimental conditions.
But rather to give him, for the first time, purchasing privileges that required no explanation to his parents.
Two days later, the platform's payment process was completed.
The 100,000 yuan special technical contribution fee entered his account after deducting necessary taxes.
With money, the gears of logistics began to spin rapidly.
The first to arrive were the hard drives.
Four heavy enterprise-grade helium drives lay side by side in anti-static bags, and even through the metal casing, one could feel that cold, hard industrial texture.
Next came the UPS, logic analyzer, and a box of bulk electronic components.
By the evening of the fourth day, that long-awaited second-hand dual-processor workstation was finally delivered downstairs.
When the deliveryman called upstairs, his mother was picking vegetables in the kitchen.
"Jiang Lin, what did you buy? They said there's a big box downstairs."
Jiang Lin came out of his room: "A computer."
Jiang Jianguo happened to be watching TV in the living room, and upon hearing this, he immediately stood up.
"Let's go, I'll go down with you to move it."
The father and son went downstairs.
In front of the first-floor unit door.
The box was even more massive than imagined; its dark brown corrugated paper shell was wrapped with five or six layers of reinforced tape by the courier company, with eye-catching black warning labels printed on the side: "Precision heavy cargo, handle with care."
The box wasn't actually very heavy, but the stairs were narrow and steep, making it hard for one person to carry.
Jiang Jianguo bent down to test the weight, said nothing, and lifted one side of the box.
Jiang Lin supported the other side.
The two of them, one in front and one behind, carried the box into the stairwell.
As they went upstairs, Jiang Jianguo panted in the back and suddenly asked: "How much did this thing cost?"
"It's second-hand, not much."
Jiang Jianguo raised his head and glanced at his son's back.
Nobody in their right mind would believe that.
Just the heavy weight and the shock-proof packaging inside meant it was definitely not cheap merchandise.
But he didn't expose the lie.
"You earned it yourself?"
"Yeah, helping a platform check data issues, consider it a technical review bonus."
Jiang Jianguo nodded.
Whatever underlying layer or data, he couldn't understand it.
But since it was legitimate money earned by his son through his own abilities, he didn't plan to ask any further.
The father and son carried the box into Jiang Lin's bedroom.
Jiang Jianguo looked at him for two seconds and said: "Being able to make money with your brain is your ability. But as the old saying goes, the body is the capital of revolution. Stay up less, and don't end up burning yourself out while taking good care of the machine."
Having said that, he turned around, pulled the door shut, and returned to the living room.
At night, Jiang Lin plugged the workstation into the power supply.
When powered on for the first time, the machine's fans emitted a low hum, much steadier than his previous second-hand computer.
Jiang Lin began to work step by step.
He used scanning tools to run a full bad-sector check on the four array hard drives, checked the SMART health information of each drive, launched a memory stress tool, and ran a high-load test continuously for half an hour without throwing a single ECC error-correction code.
Finally, he unplugged the power to test the seamless switching of the UPS.
Everything was normal.
The door to the quantitative world had cracked open for him.
But he wasn't in a rush to rush inside.
He knew very well that what he needed was not a magical furnace capable of running a few deep learning models, but an entire research system that would never deceive him.
The cleaning pipeline of the data source, the isolation mechanism of the backtesting engine, the simulation of matching logic...
Hidden traps could lurk in every single link.
He had just spent four days stripping away someone else's baseline with future functions until nothing was left hidden.
If it came to building his own system and he made the same low-level mistakes, the joke would be on him big time.
What was worse wasn't code that failed to run.
But rather code that deceived oneself, running longer and looking more like the truth.
When it truly reached that point, there wouldn't even be an opportunity to discover the error.
On the third morning, Shen Chengye sent a new email.
Subject: [Data Audit Special Task Order A-017 First Draft]
Jiang Lin opened the attachment.
This time it wasn't an internal message.
But a formal commercial contract with legal validity.
It was stamped with the electronic seal of the platform's subsidiary, and signed off by a department: "Data Compliance and Third-Party Risk Control Group."
Task Number: A-017.
Task Type: Field consistency review and missing distribution audit.
Data Scale: About 12GB.
Data Content: Three-year minute-level desensitized market quotation samples, including price, trading volume, turnover, status markers, and internal quality grouping fields.
Data Source: Historical data splicing from two underlying suppliers.
Data Status: Platform standard cleaning has been completed, but when a client reproduced a certain public factor, offline backtesting results showed instability issues.
Permission Boundary: Only desensitized data is provided; client's real strategies are not involved, un-desensitized underlying codes are not provided, and capital account information is not provided.
Delivery Requirement: Submit a preliminary audit report within seven days.
The report must include: Field consistency check.
Trading session completeness check.
Missing mechanism classification.
Consistency judgment of multi-source data splicing caliber.
List of risk items that may affect backtesting stability.
Quotation: 30,000 yuan.
Jiang Lin's gaze skipped directly over the amount at the very bottom, lingering over the words "completed platform standard cleaning".
In the engineering world, these nine characters were practically a horror story.
It by no means meant the data was very clean; it only represented that someone used a black-box script to forcibly suppress error reports.
Even more thought-provoking was the subsequent explanation: "When the client reproduced a certain public factor, offline backtesting results showed instability issues."
What are the characteristics of public factors?
Logic is transparent, code is public, and parameters are fixed.
If such a thing crashed in the client's hands, there were only two possibilities.
Either the client was an idiot who couldn't even configure the environment properly, or the data foundation was rotten.
Since the platform spared no expense to outsource the work to him for 30,000 yuan, it meant they had already ruled out the first possibility internally.
In other words, the platform washed the data once through its so-called standard pipeline, handed it over to the client, and as a result, the client crashed.
This meant that this was by no means an ordinary third-party audit task.
Rather, the platform wanted to use him—an outsider unaffected by internal politics—as a scalpel to dissect the blind spots of the platform's own cleaning pipeline.
Jiang Lin's fingertips paused on the keyboard for a few seconds.
This was an excellent springboard, but a slight misstep would lead to a bottomless pit.
If problems were found, the platform's internal face would naturally look ugly, but his technical credit would be thoroughly established, making subsequent large orders logical.
If no problems were found and he signed the report, then when clients used this set of data to run live trading with real money and got liquidated in the future—
He, an external temporary worker, would be the perfect scapegoat sacrificed to heaven.
Even more perilous was that this batch of data was spliced from two suppliers.
If the direction of the bug he found was biased and mistakenly accused a certain deep-pocketed supplier, he would likely be dragged into the legal meat grinder of the two data companies.
He scrolled the task order back down to the very bottom.
"Note: This project is the first single-item cooperation task. If the delivery result meets expectations, it can subsequently enter the Class B complete data pollution linkage audit project."
Meets expectations.
Jiang Lin chewed on these four words twice.
To get the big contract, not only did he have to pull out the bugs, but he also had to dissect the pathway of how the bugs crawled in clearly so that everyone would be fully convinced.
Jiang Lin saved the task order into a new folder.
Folder Name: [A-017]
Then he opened the terminal and created a new file named audit_log.md.
He didn't rush to write code, but instead began typing out his doubts line by line.
As an auditor, logic always precedes code.
1. Are the timestamp standards of the two suppliers consistent?
2. Before and after rights adjustment, when prices gapped, did the scaling factors of trading volume and turnover maintain strict mathematical synchronization?
3. Are the missing values randomly distributed according to symbol_id, or concentrated near specific time nodes of source_flag?
4. Could the instability claimed by the client possibly be due to a minute distribution drift in the input data caused by incremental updates?
5. Did the platform's standard cleaning process introduce secondary pollution while eliminating noise?
6. All the above suspicions must be accompanied by a minimally reproducible script that can run independently.
30,000 yuan.
12GB.
Seven days.
This wasn't his first pot of gold, nor was it some grand enterprise.
But it was a challenge scroll officially handed over to him by the Real World.
On the screen, the green cursor of the command line window beat tirelessly.
He took a deep breath, placed his hands on the mechanical keyboard, and typed the first command in this new system.
`mkdir data_raw data_intermediate reports scripts logs`
Enter.
The new system began to record the traces of its first commercial task.
🔊 Text To Speech
Listen while reading
Warning: session_start(): Session cannot be started after headers have already been sent (sent from /home/u377687657/domains/novelfull.in/public_html/header.php on line 80) in /home/u377687657/domains/novelfull.in/public_html/footer.php on line 3