How OT Capture-the-Flag Exercises Build Cyber Skills

Artificial Intelligence & Machine Learning , -Generation Technologies & Secure Development , Recruitment & Reskilling Strategy

Cybersecurity skills always grow when we practice with evidence connected to real systems. That was my clearest lesson from a recent three-day capture-the-flag exercise hosted by Dragos focused largely on blue-team forensics, network analysis and operational technology.

See Also: 28.3% of CVEs Exploited Within 24 Hours — Can Your Bank Keep Up?

I have participated in several CTFs, including the National Cyber League and MetaCTF. The Dragos annual exercise felt different because the challenges had a context that is fundamentally alien to those who have no experience working in operational technology. Though much of the work involved more common techniques such as Wireshark PCAP analysis, the context of the analysis contributed to the difficulty

That distinction matters. In a traditional cybersecurity CTF, many experienced practitioners already understand the protocols, file formats and tools in front of them. OT changes the equation. Someone can know cybersecurity well and still have little experience with the protocols and devices used in industrial environments.

Modbus is a good example. A security practitioner may understand networking, packet analysis and lateral movement but have little reason to encounter Modbus in daily work. In OT, it may be central to understanding how controllers and other devices communicate.

The technical evidence only becomes useful when you understand its context.

The Hard Part Is Understanding the Story

The Dragos CTF included multiple scenarios involving environments such as utilities, water systems and power facilities. Participants received network traffic and other artifacts, then had to find flags by analyzing the evidence locally.

Wireshark was one of the tools I used throughout the event. The challenges exposed me to Modbus/TCP, BACnet, HTTP authentication, SMB and SSH. I also used CyberChef to decode data and Impacket to examine a provided Windows SAM database.

But using a tool is not the same as understanding what its output means.

A Modbus register write becomes more significant when you know which controller received it, what equipment the controller affects and where the activity fits in the attack sequence. A packet capture is not just a collection of packets. It is a partial record of a story.

One useful artifact was a hand-drawn network diagram connecting remote access and supporting systems to a Wonderware HMI, PLCs and other devices. The diagram helped turn individual packets into an attack path.

That is the type of context defenders need during an actual investigation. They have to determine what happened, how the attacker moved and what the activity meant to the environment.

OT makes that reasoning especially important because technical actions can have physical consequences. But the broader lesson applies outside industrial environments. Every organization has systems with dependencies, business purposes and operating requirements. Investigators make better decisions when they understand those relationships before interpreting the evidence.

A Beginner and an Expert See Different Problems

The exercise also reinforced how differently beginners and experienced practitioners approach the same evidence.

For someone with experience, opening a PCAP file may feel trivial. A beginner may not know what a PCAP file is, which program can open it or what information inside matters.

That gap is easy to overlook when we teach cybersecurity.

An experienced analyst who receives a packet capture may immediately ask which protocol is relevant, what sequence of events occurred and what the attacker was trying to accomplish. A beginner may first have to determine how to open the file.

Neither step is unimportant. The beginner who learns how to open the evidence, filter traffic and recognize a protocol has already gained useful experience, even without capturing a single flag.

As I said during my discussion the exercise, "you need to get your hands dirty."

That is why I would still describe this CTF as approachable for an entry-level participant. The early challenges provide an opportunity to learn the mechanics. The difficulty then ramps up as participants progress.

The progression itself created another challenge. Unlike CTFs that show every problem from the beginning, this event unlocked challenges sequentially. You could not see the question until you solved the current one.

That structure makes troubleshooting part of the exercise. If you get stuck, you cannot simply skip ahead to a problem that better matches your strengths. You have to work through the obstacle before moving forward.

Experienced practitioners may recognize familiar patterns and move faster. Beginners have to build those patterns for the first time. That experience is valuable because real incidents rarely arrive as neatly labeled classroom exercises.

Why a CTF Teaches What a Textbook Cannot

Traditional instruction still matters. Students need networking concepts, attack models, protocols and defensive theory. But knowing what should happen is different from knowing where to begin when somebody gives you raw evidence.

CTFs force participants to make that transition.

You may know how an attack progresses in theory. In an exercise, you have to identify the relevant evidence, choose a tool, filter out noise and reconstruct the sequence yourself.

That work also happens under some pressure. You may not know whether your first assumption is correct. The obvious tool may not provide the answer. You may have to decode data, reconstruct traffic or try another approach.

That type of troubleshooting is difficult to reproduce through reading alone.

For a new cybersecurity graduate, the biggest lesson may be learning where to start. For an experienced practitioner, the benefit may be exposure to an unfamiliar technology or protocol. OT CTFs can provide both because even people with strong enterprise security backgrounds may have little exposure to industrial communications.

Ten years ago, finding an accessible OT exercise of this type would have been much harder. Today, practitioners can gain exposure to these environments without first working inside a utility or industrial facility.

AI Can Help - and It Can Take Away the Lesson

AI tools are adding value to many cybersecurity processes these days. But it adds a new wrinkle to CTFs.

For straightforward questions, an AI system may identify a flag or explain a protocol almost immediately. Used carefully, that can help someone learn an unfamiliar file format, understand a Wireshark filter or identify a sensible step.

But that convenience carries a cost.

If AI solves all the easy problems for a beginner, the learner may arrive at a difficult problem without having built the skills the earlier challenges were intended to teach. The person has progressed through the CTF, but the person's knowledge has not progressed at the same pace.

The harder challenges also expose weaknesses in AI reasoning.

A common CTF tactic is to take a malicious PowerShell command, encode or transform it and split it across multiple packets. A person who has seen the technique before may recognize that apparently meaningless data needs to be reconstructed and decoded.

An AI tool may instead treat the fragmented data as gibberish or conclude that nothing useful can be recovered.

That creates a dangerous combination: a confident answer and a user who lacks enough experience to challenge it.

AI can therefore help experienced practitioners move faster because they have enough context to judge the response. Beginners face a different risk. If they ask AI to do the reasoning rather than help with the reasoning, they can bypass the exact struggle that makes the exercise valuable.

I found that distinction especially important in this CTF. The difficult questions rewarded pattern recognition. You had to know that apparently unrelated artifacts could fit together and that an encoded fragment might have another layer underneath it.

AI is useful as an assistant. It is less useful as a substitute for curiosity, troubleshooting and skepticism.

Context Is the Skill I Want to Keep Building

The strongest lesson I took from the exercise was not Wireshark, Modbus or any particular forensic tool. It was context.

Cybersecurity work is not simply recognizing malicious traffic or knowing the correct command. Investigators have to understand what systems do, how they communicate, what an attacker appears to be attempting and what the consequences could be.

A CTF compresses that experience into a controlled environment. Beginners learn how to approach unfamiliar evidence. More experienced practitioners discover gaps in their own knowledge. Both groups practice making decisions without having every answer laid out in advance.

That is difficult to teach from a slide deck.

The best exercises make us reconstruct the story ourselves. They give us enough evidence to begin, enough uncertainty to make us think and enough difficulty to make failure useful. That's why I enjoy doing them.