Your Tooling Is Built To Catch The Attacker Who Wants To Be Caught
On an emulation engagement we held domain administrator rights for eleven days and nobody asked a question about it. Domain administrator is the account that owns every machine joined to a company's network, so once you hold it the game is effectively over. We did not use malware to get it and we did not use any after. We logged in during business hours from a workstation their own administrators used every day, and we ran the same commands those administrators ran, using tools that shipped with the operating system. When we needed to put a file onto machines across the environment, we used their own patch deployment system. That system exists to copy software onto every computer in the company and run it, so once we held administrator rights over it we pointed that same machinery at the machines we wanted. The file went out looking like a routine patch leaving a trusted server. It landed in the one folder on each machine that the vendor formally advises security teams to exclude from antivirus scanning, so nothing inspected it on the way in.
In the debrief their security operations center, the team that watches alerts around the clock, pulled up a dashboard showing a mean time to detect of fourteen minutes. That number was real. Every tabletop exercise they had run for two years was a ransomware scenario, and against ransomware fourteen minutes is a strong result. The reason it did nothing for them against us is that we never behaved like ransomware. We had been in the environment eleven days before a helpdesk ticket about a slow file copy put someone onto us.
Ransomware has to make noise to get paid
Ransomware is malicious software that scrambles an organization's files and holds the decryption key for payment. The business model only closes if the victim finds out, so the criminal has to encrypt enough systems to hurt and then announce himself. Making noise is part of the job rather than a mistake he made. The faster and louder he moves, the less time you have to pull backups offline, which is why that tradecraft has gotten more aggressive over the years instead of quieter.
A normal security program gets built around that adversary without anyone deciding to do it. The alerts are tuned to fire on unusual programs running and on large numbers of files changing at once, which is what encryption looks like from the outside. The tabletop exercises start their clock the moment a ransom note shows up on a screen. A detection vendor sells against that same timeline, and when a board asks how the program is doing, it tends to ask the question the vendor already answered. Cyber insurance underwriters want to know how fast you would detect and contain an encryption event before they renew a policy. Each of those habits assumes an attacker who is trying to get noticed, because a ransomware crew has to be.
The Silent Train relay network
At the end of October, Black Lotus Labs, the threat research arm of Lumen Technologies, published findings on a framework it calls Silent Train. The operator is Volt Typhoon, a Chinese state group that United States agencies have been warning about since 2023 for burrowing into critical infrastructure and then sitting there.
Silent Train is not a botnet in the way most people picture one. A botnet is a standing collection of hijacked machines that an operator keeps and reuses, which gives it a shape defenders can map and eventually take down. This has no standing shape. It is a rotating supply of compromised home and small office routers, internet connected devices, and enterprise servers, chained together to carry a single operation and then released.
Researchers watched a chain get assembled in under two minutes, and a typical session lasted about four hours. A compromised home router was used for one session and then restored to its original state, so the evidence erased itself. At any given moment the count of live nodes sits near zero, which is why the researchers were careful to say the activity cannot be measured by size and that what exists today will be different tomorrow.
The routing defeats the controls most companies lean on. Traffic starts from rented servers outside the United States, moves through compromised devices overseas, and exits through a compromised device inside the United States before it ever touches the target.
Researchers counted as many as six hops. By the time a login reaches your network it arrives from an American residential address with no bad reputation attached to it, so geographic blocking does nothing and threat intelligence feeds have nothing to say about it.
Volt Typhoon also went after control plane infrastructure. Control plane is the layer that manages other systems rather than doing the work itself, which covers endpoint management platforms, remote monitoring and management tools, identity systems, and the orchestrators that configure wide area networks across every site a company runs. They reached those with previously unknown vulnerabilities in products including Versa Director, N-able N-central, Cisco Catalyst wide area networking, and VeloCloud Orchestrator. Compromising one of them reaches every network and every administrator underneath it at once. The researchers were explicit that no lateral movement was required. Lateral movement is the noisy business of hopping from machine to machine, and most detection logic is written to catch exactly that. Owning the management platform meant they never had to do it. Cisco's own researchers reported the same week that a related operator used one of those orchestrator flaws to install a malware framework directly onto wide area network controllers.
Across three years, Black Lotus Labs notified hundreds of victims. They sat in water and energy utilities, in transportation, maritime, rail and aviation operators, in emergency services, telecommunications carriers, satellite providers, defense contractors, and universities with military ties. The three states that came up most often were Hawaii, California, and Texas, which is where the military installations, defense manufacturing, satellite ground infrastructure, and logistics supporting Pacific operations happen to sit.
How they work once they are inside
A joint advisory from the Cybersecurity and Infrastructure Security Agency and partner agencies, catalogued as AA24-038A and published in February 2024, is still the most specific public account of how this group behaves inside a victim. It says they held access in some environments for at least five years. In one case they pulled the Active Directory database, the file holding every password hash in the company, off three domain controllers across four years. They did it repeatedly over that span to keep the stolen credentials current as employees changed passwords.
What they used is ordinary administrative tooling. vssadmin to make a shadow copy, ntdsutil to grab the database, and a built in Windows library called comsvcs.dll to dump credentials out of memory. For everything else, cmd, PowerShell, netsh, certutil, nltest, reg save, and the standard net commands. All of that ships with Windows and gets run by real administrators every day for legitimate reasons. Working this way is called living off the land, and it defeats an entire category of defense, because a tool that decides whether something is malicious by inspecting a file has no file to inspect. Behavior becomes the only available signal, and the behavior is identical to an administrator doing his job.
They were careful about the small things that usually give an intruder away. Because they used valid accounts, there was no spike in failed logins for anyone to notice. They worked normal business hours, so nothing stood out at 0300. They did not wipe logs, since an empty log is an alarm by itself, and instead removed only the entries that pointed back at them. Their traffic came in over the company's own remote access and looked no different from an employee working from home. In one victim the whole thing started with a domain administrator password left unprotected on a network appliance. They also pulled a network administrator's browser history and saved passwords, and collected the personal email addresses of the information technology staff, which gives them a way to approach those people directly later.
The case that detection does catch this
The fair pushback is that good detection does catch some of this. Modern endpoint detection and response, which is software that watches how a machine behaves rather than scanning files for known bad ones, will flag part of what I just described. Running ntdsutil on a domain controller is a high quality alert, and a mature team with good analytics will catch it. Anyone who runs a serious detection program knows that, so saying living off the land is invisible would be too strong.
The problem with that is the baseline it depends on. Detecting this kind of activity means knowing what normal looks like on your own network, and almost nobody does. Having an alert for ntdsutil is not the hard part. The hard part is being able to say which administrators are supposed to run it, from which machines, and on what schedule, so that a different person running it on a Tuesday afternoon actually stands out. Without that baseline, an analytic either floods the queue until somebody turns it down, or it shows up turned down on the first day. We have watched both happen. The second is more common, because an analyst who gets paged forty times a week over legitimate administrator activity stops reading the pages.
The control plane finding makes this harder in a way most programs have not absorbed. Your endpoint detection tool is deployed, configured, and updated from a management console. An adversary who owns that console sits upstream of the sensor, holding the authority to change what it watches and where it reports. The same applies to the remote monitoring platform and the identity provider. Those systems tend to get treated as plumbing, and the access lists on them are usually older and longer than anyone remembers.
What the access is for
The federal advisory assesses with high confidence that the purpose of this access is to enable movement from the business network into operational technology, the computers and controllers that run physical equipment rather than handling email and files, in order to disrupt those functions. Investigators found the actors had tested access to operational systems joined to the corporate domain using default vendor credentials, the passwords equipment ships with that nobody changed. In one confirmed compromise they reached a control system and were positioned to reach a second.
Investigators also looked at what the intruders had saved on the machines they held. Stored remote session profiles, which are the saved connection shortcuts an engineer keeps so he does not retype an address, pointing at water treatment plants, water wells, an electrical substation, and the network and security devices in between. An engineer had built a convenient index of the physical plant and the intruders inherited it. The advisory notes that in some cases these actors could reach camera surveillance systems, and that the access they held would support manipulating the cooling in server rooms and interfering with energy and water controls.
This is a different objective from espionage. Collecting intelligence means taking documents and leaving, and the access described in that advisory is built to stay. It preserves the ability to interfere with physical equipment at a moment the operator picks, and that timing would follow a political calendar rather than a commercial one. Lumen's researchers landed on the same assessment of purpose, which was pre-positioning access that might be wanted during a period of geopolitical tension or an outright crisis. An operator working toward that has no payment to collect and no deadline, so he can wait out your staff turnover and the thirty days of logs you keep.
What you can check without buying anything
Start with your own detection number and ask what it was measured against. Pull the last two years of tabletop exercises and incident simulations. If every one of them opened with ransomware, encryption, or a ransom note, then your mean time to detect is a ransomware number and it says nothing about a quiet intruder. Run one exercise where the scenario is a valid administrator account being used correctly, during business hours, for ninety days, and find out what in your program would have raised the first question. Write the answer down even if the answer is that somebody would have caught it by luck.
The baseline is next and it is harder than it sounds. Pick a couple of administrative actions that an attacker needs and a real administrator also performs, such as copying the Active Directory database or dumping process memory. For each one, name who legitimately does it, from where, and how often. If your team cannot answer in an afternoon, no behavioral analytic you buy will answer it either, because the product depends on the same baseline you do not have.
Nobody keeps the control plane on an asset list as what it actually is, so build that list deliberately. Every system that can install software, change configuration, or push policy to a fleet of machines goes on it. That means the endpoint management platform, the remote monitoring tool your managed service provider logs into, the wide area network orchestrator, the identity provider, and the patch server. Then count the accounts that can log into each one, check how many of those people still work there, find out whether any of them can reach it directly from the internet, and ask whether a login from an American home internet address would look wrong to anyone on your team. The research answers that last question for you, because an American residential address is exactly what the connection looks like on arrival.
The path from your business network to your physical equipment is the part that goes unmapped, and walking it usually produces surprises. Find out which operational devices are joined to the corporate domain, since a stolen corporate password opens those directly. Check whether any of them still carry the credentials the vendor shipped. Open the saved session profiles on your engineers' workstations and on the machines they use to reach sensitive systems, and read what they point at, because that list is a map of your plant sitting on machines a phishing email can reach. Building systems belong in the same walk. Cameras, access control, and server room cooling are all operational technology, even though facilities owns them and the security team usually does not have them on any diagram.
Last, look up how long you keep your logs. Thirty days is common, ninety is better, and a regulated few keep a year. The documented dwell time for this adversary is five years. If they entered your environment in 2022, the record of how they got in and what they touched aged off disk before anyone went looking, and no amount of threat hunting recovers data that is gone. The federal advisory points defenders toward a specific set of application log entries, the ESENT events numbered 216, 325, 326, and 327, because those tend to survive longer than security logs. Pull your retention figure and your longest assumed dwell time and set them side by side, because that comparison decides whether an investigation is possible at all.
What we do with this
We run active threat emulation, and for this scenario we run the quiet profile rather than the smash and grab. We take the access and then behave like someone who intends to keep it for years, using your own tools, your own accounts, and your own management consoles. At the end you get the full record of what we did set against the full record of what your program saw. In most engagements the distance between those two documents is wider than the security team expected, and the useful part is seeing exactly which controls should have fired and why they did not.
On the advisory side we review detection strategy against the adversary that fits the organization instead of the one in the vendor demo. That means building the baseline of normal administrative behavior every analytic depends on, and getting the control plane treated as the highest value target in the environment rather than as infrastructure.
The physical side is the half most firms never reach. We map the real path from the business network to the equipment, the operational devices sitting on the corporate domain, the default credentials still in place, and the building systems nobody has on a diagram. Then we work out what an adversary holding that access could do to a specific site, and what it would take to see him doing it.
This gap persists because it falls between two trades. Detection engineering and facility security get bought separately, from different vendors, by different parts of the company, and nobody is tasked with following a path that begins at a credential stolen four years ago and ends at a pump or a substation breaker. We work that path from both ends.
We're based in Dallas, Texas, and we go wherever the problem is. Red Cell Security provides cybersecurity and physical security consulting to clients across the US and around the world.
Comments