CyberCompare

“What are you going to do with that skillset? No one cares about security!” | Expert interview with Dale Peterson, part 1/2

About our interview partner

Dale Peterson

Dale Peterson is the co-founder and managing director of Digital Bond Inc. He has many years of experience in the IT security sector and is the creator and program chair of S4 Events, the world’s largest conference for industrial cyber security.

Hello Dale, it’s really an honor to have you with us today. You now have about 37 years of experience in all kinds of security related matters, having started your career path at the NSA, and you’re now supporting the industrial security community for more than 20 years. Many of our audience know you and are also fans of your podcast or reading your weekly newsletter. But can you maybe share some of the most memorable moments of your career with our audience?

Well, it, it has been a long time, 37 years, but I would say most of the memorable moments are things that just happened, lucky that I stumbled upon. I tend to find things that I’m interested in and jump into them, whether it makes business sense or not.

So for example, I went to school at university of Illinois and was graduating back in 1984 with a finance degree. And I was going to be an actuary, someone who determined insurance rates. But I had read this article that the state department, what most of the world calls a ministry of foreign affairs, had a really interesting test to be a diplomat. So I went to sign up to take that test because that sounded much more fun than being an actuary. And they said, no, you’re too late. That’s closed. But by the way, there’s another agency called the national security agency, NSA, and they have a test. Do you want to take that one? And back in 84, no one really knew who or what NSA was. So, you know, shrug my shoulders. And I said, sure. And I took the test and it ended up, I scored high enough to be a crypt analyst, so I had skills to be a code breaker. So after that I got a job offer.

And, and remember this was 84 and my dad said, what are you going to do with that skillset? No one cares about security! This is a dead end job. You have this actuary thing. It’s a sure thing. You should do that. But I just said, Oh, this sounds like fun. It sounds interesting. And I did it and things worked out and obviously security became a big field and it worked out for me.

And the same thing sort of happened with ICS. In 2000 we were a small company, four people doing consulting, security consulting and some water utility out in the Western United States contacted us for some reason. I still don’t know why. And they said, can you do an assessment of our SCADA system? Okay. I had no idea what SCADA was. We needed the business. So being a normal consultant, I said, sure.

We ended up competing against IBM and somehow won the contract. And I went out there and I just fell in love with the field. I mean, instead of being in a data center, we were out looking at pumping stations and things like that. And I just decided, this is what I want to do. And again, back in 2000, there wasn’t much of a future for ICS security. There was no one who really cared about it. I was surprised these people even cared about it, but I just stumbled in there.

So I won’t say that it’s always been successful. I’ve stumbled into things like banking security, which I really didn’t like. In the late nineties, we tried to develop a product for internet brokerages that flopped. But I would say just doing things that I really enjoy and find interesting has led to a lot of the highlights.

What are maybe some of the mentors on your journey that you’ve met?

That’s kinda tough because there weren’t a lot of people doing what I do. So a lot of times it was sort of trailblazing. Most of my mentors weren’t actually people, but they were more books and speakers and concepts. So for example, when I wanted to get into ICS security, I leaned heavily on Tom Peters, who at the time was one of the first pushing this “brand you” concept. And he talked about how do you develop a personal brand? So I got his book, and there were 50 things you could do to make your name in this field. So people like that are more kind of who I learned from or who helped me out along the way.

What was one of the most difficult technical challenges which you encountered in your field and how did you and your team address them?

Well, if you’re talking pure technical, probably the hardest thing we did was back in 2006. We got a research contract from the Department of Homeland security to develop the first intrusion detection signatures for ICS protocols. And some of the protocols like DNP3 and Ethernet/IP required preprocessors to deal with the packets that came in, not in one packet, but were split between multiple packets and were involved in multiple levels of the stack.

So I hired a really smart guy, Daniel Peck, and he banged away at this problem for a month or two and just could not solve it. And we were on the hook to deliver. And we tried and tried and tried and we failed.

And then we finally we said, well, let’s get someone who has written one of the most complex pre processors before. So we hired him as a consultant. The first thing he told us was, Oh, all the documentation is wrong.

And this was true. Even years after this, like 2011, 2012, it was still just wrong. So if you did exactly what the documentation said, it would fail, but this guy said, this is how it really works. And Daniel finished it within a week.

So that was specific, but one of the real technical challenges now for every asset owner is: We’ve got limited resources, we’ve got a ton of things we can do. Where do we put our resources?

And the criteria I always use is efficient risk reduction. Where do we get the most risk reduction for the next dollar we spend or hour we spend on this. And that’s, I find that to be a very interesting problem. And it usually bends people’s minds a little bit because a lot of times it’s not the things that they would think it’s, it usually is something else. And that’s a challenge at every company.

What are some of the unique approaches, which you use to define risk, for example, together with your customers?

The key is you really have to focus on consequence. If I’ve got so many things to do, the real key is to say, if I implement this good security practice, do I actually reduce my risk? And the only way you can understand that is if you understand the consequence of an attack and would this reduce the likelihood or the consequences. And you have to determine what your priority is.

We’re seeing more and more cyber process hazard analyses now. So I think the industry is finally coming to grips with that.

What kind of current developments in industrial cyber security do you find interesting?

Well, this is what I do. Our S4 event is looking for those sorts of things, looking for the bleeding edge, what’s going to come in the future. Our tagline is even “create the future”. I probably only spend about a third of my time consulting now. So I could give you a long list, but I think for your audience, what’s most interesting are new security developments around level 1 – so PLCs and controllers, both what’s becoming available now and what we’re going to see in the next one to three years.

Because if you think about this, one of the real challenges in securing your system is if the adversaries are inside the perimeter, that they can go straight to level 1. If they have the engineering and automation skills, they don’t need to compromise anything else because those systems are “insecure by design”.

Everything you want as an attacker is a documented feature. You want to upload new firmware, find the upload firmware command. If you want to change the program or the recipe, just find the command to do that. You don’t need to hack anything.

So finally getting past that is going to make a major difference in this ability to protect your system. Some of the leading vendors now are actually offering signed firmware. That’s just huge. So you get away from this thing where bad guys just create bad firmware, upload it, to break devices. So if you could only upload signed firmware from the vendor, that’s a huge security improvement. Those are things that can be put in place now by for a lot of systems.

And we’re starting to see encrypted and authenticated protocols. So like Modbus Secure, PROFINET wrapped in TLS. So we’re finally seeing those, which is really exciting. That’s short term.

Longer term, what I’m even more excited about is the idea of virtualizing level 1. So everything but the IO being virtual. And once you get to that point, now you can tear those down or put them up in in minutes rather than months. And that just changes the whole ballgame.

The same thing is true on these level 1 devices, which are basically computers. And we’ve seen that Emerson, for example, has a controller that’s virtualized. Eventually for security, reliability, maintainability and performance reasons, you’re going to see those more. In the US, most of the larger control systems that are deployed now are actually virtual platforms at level 2. And that’s going to move to level 1.

Many of our customers are of course  approaching us with questions on cloud connectivity and how to do this in a secure way. So what are your thoughts on this?

Well, this is actually where I’m spending much of my time now. I think the best way to look at this was explained by Bryan Owen during our S4 event 2019. He broke it down into open loop and closed loop, because this would be a way for the automation world to understand it. So he says open-loop cloud services, you basically send data to the cloud and they process it and they don’t send it back to the control system.

You know, maybe you look at it through their web portal. Maybe they send you periodic reports or something like that. You just deploy a one-way device, the data diode, a unidirectional gateway. And the good news is the prices on those are going way down. Those were really expensive for a long time, you know, hundreds of thousands of dollars to buy one with a few protocols. But now I think I saw one that was a thousand dollars for a small installation now.

And, and I think one of the things you’ll see then is, it’s a no brainer. You say, I want these cloud services. I want predictive maintenance, efficiency, studies, things like that. But I don’t want any risk to the integrity or availability of my control system. Simply put in one of these one way devices it’s done. So that one’s real easy, where it gets tougher as the closed loop.

So now imagine that you want the cloud service provider to send either information or commands back to your control system. And this is where I don’t think I’ve seen yet a good solution. I think I know what the solution should be, but I haven’t seen anyone do it yet. Which what we need to see is if you look at like AWS or Azure, they have these edge devices that will control the communication to the cloud, the authentication, the encryption, the rights to the cloud and such, and you definitely need an edge device. I’m really amazed at the number of people offering these closed loop cloud services without edge devices today, they’re essentially going server to server for VPNs. And they’re saying, trust us. So that’s half the solution, the other half of the solution, but it’s in the same edge device is a deep packet inspection firewall or gateway, like a Tofino or M-Guard, a number of people have these. It can actually say, this is what’s allowed to come in.

Important is that the asset owners need to demand this. And we haven’t seen that yet from the asset owners.

You can find the second part of this interview here.

Are OT and IoT security issues for your company? As an independent entity with a portfolio of proven security providers, CyberCompare can provide you with comparative offers at no charge and with no obligation. Reach out to us or use our diagnostic to learn more about your cyber risk profile.

Please remember: this article is based our knowledge at the time it was written – but we learn more every day. Do you think important points are missing or do you see the topic from a different perspective? We would be happy to discuss current developments in greater detail with you and your company’s other experts and welcome your feedback and thoughts.

And one more thing: the fact that an article mentions (or does not mention) a provider does not represent a recommendation from CyberCompare. Recommendations always depend on the customer’s individual situation.