"A commission formed to offer advice on cybersecurity to the next president is nearing the completion of its work, and some of the recommendations are likely to conflict with elements of President Bush's Cyber Initiative. It will be finalized very shortly," said Rep. Jim Langevin (D-R.I.), co-chairman of the bipartisan Commission on Cyber Security for the 44th Presidency. "The findings are preliminary at this point." The commission, created in November 2007 by the Center for Strategic and International Studies (CSIS), held a series of public meetings to hear recommendations on issues of information security, identity theft and government leadership. It plans to present its findings to the new president prior to his inauguration in January. When it does, one of the biggest departures from current cyber security policy will be the commission's recommendation to take the lead away from the Homeland Security Department and give it to the White House."

The White Paper on Industrial Control Systems, of which I was the principal author, was prepared for this effort. I would like to make absolutely clear my support for the work being performed by the Congressman’s Committee and the recommendations they have prepared.

SEL, Emerson, Invensys, Honeywell, and other major vendors are taking security more seriously. They are incorporating security technologies and providing security consulting services. The key will be end-user acceptance.

SANS – "Some Good News for a Change. American utilities have made a 180 degree turn in the past five months - - no longer trying to claim that their control systems are ‘safe from cyber attacks.’ As a result, oversight organizations (like NERC, North American Energy Reliability Corporation) have stepped up to help them measure the effectiveness of their security using the right metrics, and are reaching for consensus on what must be done to secure the systems and how utilities can be sure they have done the right things..."

There is a need to develop the right metrics. To date, the NIST standards are the closest to "the right" standards. When public in 2009, I believe the NRC Regulatory Guide DG-5022 will be the most appropriate guidance with metrics for all industries.  

On October 28, US CERT issued Critical Infrastructure Information Notice- CIIN-08-302-01, ICONICS Dialog Wrapper Module ActiveX Control Vulnerability. It stated: "In January 2007, a buffer overflow vulnerability in the ICONICS Dialog Wrapper Module ActiveX Control was discovered. This vulnerability may allow a remote attacker to execute arbitrary code on a vulnerable system. Exploit code for this vulnerability was made publicly available on September 21, 2008…"

The January 2007 buffer overflow vulnerability was on the website demo, not in the released software itself, and was explained, and repaired, at least six months ago. This is simply another reason for the need for a non-governmental CERT for Control Systems with control system expertise.   

Joe Weiss

There has been a lot of discussion on the SCADA listservers about cyber vulnerabilities of control systems. What I want to address is that cyber vulnerabilities or cyber security for that matter does not equal CIP. Cyber security is a subset of CIP.  There are other aspects, including physical security, fragility of the asset, the availability and expense of backup support, etc. Many of the control system cyber events that have caused actual equipment damage or directly shutdown equipment have not been because of traditional cyber vulnerabilities. The corollary is that traditional cyber vulnerabilities (viruses, worms, buffer overflows, etc) have caused “burps” in the system shutting down systems, but so far have not caused equipment damage. That is not to say they can’t as the demonstration at the August ACS Conference illustrated. To demonstrate this premise, I will only address events that are publicly acknowledged. The Bellingham pipeline rupture, Browns Ferry and Hatch shutdowns, Maroochy wireless hack, the Florida outage, and a recent automatic shutdown of a nuclear plant were all CIP system-related events that had cyber “overtones”. Worms such as Code Red, Slammer, and Blaster affected control systems by directly or indirectly shutting many down many Windows-based HMIs but did not damage field equipment. System issues are generally architectural and policy-driven not technology-driven.

One specific issue has reappeared that needs to be addressed - the disconnect between power plants and the plant switchyards. The switchyard (inside the plant fence) is generally “owned” by the power plant (nuclear, fossil, or hydro). Plant staff are very knowledgeable about the equipment inside the plant but often are not technically cognizant about switchyard equipment (breakers, relays, etc) and therefore rely on others such as corporate T&D or local electrical contractors to maintain these systems. This creates the possibility of inadequate oversite of the switchyard equipment. Last year, one utility did have an automatic nuclear plant shutdown because of inappropriate testing of plant switchyard relays. (I believe there have been numerous fossil plant shutdowns because of relay testing but they do not have to be reported.) Even though I  have not been able to confirm the event had a direct cyber aspect, it could be cyber-related since upgrades to plant relays will include remotely accessible relays. What adds to the concern and disappointment is that most utilities have defined their plants as not being NERC critical assets and consequently this potential common cause failure, and others including Aurora, are not being addressed.

 According to the new “Global Threat Report” from ScanSafe, “…energy companies worldwide have a nearly 200 percent rate of being hit with Webborne malware attacks. According to the report, energy companies experienced more Web-based malware attacks than any other vertical market in the third quarter of this year, with an increased rate of exposure of 189 percent.” I had a chance to discuss this report with the author since almost 4 years ago, Riptech made similar statements.  My concern is that most companies doing log-management are only looking at the corporate firewalls and DMZs. Consequently, they have no view into the control systems. It is not to say the report is wrong, it is just to say it hasn’t addressed what is happening with the control systems. 

The following blog discusses using control system software to distribute malware: http://carnal0wnage.blogspot.com/2008/10/malware-targeting-industrial control.html 

”Malware targeting industrial control software(?)

So this morning I was doing my usual malware roundup and looking for anything new or vaguely interesting. Lots of the usual sites all serving the same thing. The pdf exploit (I've a special fondness for this one, see the pentesting failures post), the snapshot_viewer_activex exploit, ms08_053, realplayer11, ms08_011, ms06_014 and a few others. All pretty popular right now.

Then I saw a site that has been floating around serving up malware for a while now. It's been up for about a year I think. It's always had a nice index.htm page with a list of iframes serving up all of the above and some others. I generally have a quick look every now and again and find it's always the same stuff. Lots of reuse of exploits, etc...

Today was a surprise as I found something 'new'. The page has another exploit added. Nothing new about that but it's what the exploit is for that is surprising.

hxxp://www.wackystone.com/counter/IConics.htm

In August a stack overflow exploit in the Iconics Vessel ActiveX control was released. The exploit is in the dlgwrapper.dll [Dialog Wrapper Module ActiveX control]. Tebo and kf wrote a Metasploit exploit module for it. [http://www.milw0rm.com/exploits/6570].

Iconics makes plant automation software for various industries including oil, gas, pharma, airports, etc... SCADA anyone?

A quick decode of the ucs2 encoded payload reveals:

hxxp://www.wackystone.com/counter/taskmgr.exe

The exploit downloads taskmgr.exe, a dropper that installs a second stage piece of malware. I've not downloaded that as yet so I don't know the actual payload or it's function.

I guess what is interesting to me is that the malware authors have decided to use an exploit that has a somewhat small target audience. I could be wrong as I'm not that familiar with those industries and perhaps the software is really widespread.”

Posted by Dean De Beer

Joe Weiss

The SCADASec listserver continues to be unsettling to say the least. There are still multiple definitions that have no uniform meaning inside or outside the control system cyber security community. These include the terms IT, security, SCADA, etc. There have been numerous attempts to standardize definitions (ISA, NIST, PCSF, IEEE, etc), yet it still a work in progress. ITAA and other IT organizations are not part of this process. If you take a high enough view, all computing systems can be construed as part of Information Technology. It is the same with cyber security policies and procedures. At the highest level, the general requirements are the same - minimize unsecured remote access and take responsibility. Unfortunately, the devil is in the details. That is why ISO-17799, 27001, etc are not sufficient to address control systems.

 I would like to give kudos to Kevin Finisterre. On 9/16, he wrote:
This is kinda cool...

http://www.selinc.com/relarack.htm

“Welcome to the SEL Relay Test Rack. Connect to real SEL relays from any computer that supports TELNET communications. When you click on the Telnet links below, your web browser will start   a "Telnet" session to an SEL Communications Processor. Through the   communications processor, you will be able to connect to one of   several different SEL relays, listed below.”

Interestingly enough, the SEL relay test rack website now has the following message:

“The SEL Relay Rack is temporarily unavailable.  We apologize for any inconvenience.”  I think this is the kind of public disclosure that can help industry by pointing out information that should not really be in a public venue. There are other more secure ways to get this information to the appropriate audience.

An indication of the convergence of the financial mess and the lack of control system cyber security understanding is a Wall Street firm’s response to the SCADASec listserver.  They have real-time data connections into power plants yet the financial firm’s management doesn’t yet see it as an unacceptable risk that needs to be addressed (sound familiar).

The electric and water industries have made progress in IT security and IT-like systems (modern SCADA and DCS). However, I believe progress in addressing field control systems in power plants and substations is still abysmal. There are no publicly available statistics about the number of power plants and substations that have been identified as NERC critical assets. FERC and Congress obviously are not satisfied with the progress in addressing general lack of addressing power plants (including Aurora) as can be seen from the Congressional hearings. As an example, SERC serves as the regional entity with delegated authority from NERC for proposing and enforcing reliability standards (including the NERC CIPs) within the Southeast Region. SERC includes the states of North and South Carolina, Georgia, Alabama, Tennessee, Louisiana, Kentucky, and parts of Texas, Oklahoma, Illinois, and Virginia (based on the SERC website map). There is not a single power plant (fossil or nuclear) in the SERC region considered a NERC critical asset despite the number of large nuclear and coal plants.

Friday I had a conversation with an individual from one of the largest coal plants in the country (not in the SERC region). In fact, this plant is larger than any individual nuclear unit. However, it is not a NERC critical asset. I have also been working with another utility with very large coal plants (again not in the SERC region). The multiple units make it one of the larger generating stations in the country. They are also not NERC critical assets. At the Electric Power 2008 Conference, a number of power plant managers talked about how their units were not NERC critical assets. In fact, one went so far as to say they are no longer a black start unit just to get away from the NERC CIP requirements. The past two weeks’ blogs described the abysmal (there is that word again) efforts of the water industry to address control systems. What does it take to get utilities to take security seriously?

The CSIS Industrial Control Systems (ICS) White Paper providing cyber security recommendations to the next presidential administration will be published in full. The actual CSIS report (48 pages long) will have a condensed ICS discussion with recommendations (approximately 700 words). There was a an additional comment in the shortened version thanks to input from an international colleague: “A number of North American control system suppliers have development activities in countries with dubious credentials (e.g. a major North American control system supplier has a major code writing office in China) and a European RTU manufacturer has code written in Iran” - so much for security by obscurity. The condensed discussion was provided to CSIS for comments this week. I will provide the final version once I have the comments from CSIS.

The SCADASec listserver is not meant to be a training course in control system cyber security. There are very few places to find credible control system cyber security training. This week, ISA will also be providing control system cyber security training in Houston and I will be giving a lecture on industrial control system cyber security at Mississippi State University.

Joe Weiss

I wanted to provide my insights on the IDC white paper: Critical Infrastructure Cybersecurity: Survey Findings and Analysis. It provides findings and analysis of a survey of 199 responders conducted by Secure Computing regarding critical infrastructure cyber security.  The inaccuracies and inconsistencies I point out are not with the Secure Computing survey, but with the IDC analysis and interpretation of the results that are presented in the report.   “Finally, the survey respondents were asked what they believed was the biggest bottleneck to critical infrastructure security. The largest number of respondents believed that cost was the biggest bottleneck.  Apathy was cited as the second biggest bottleneck.  Government bureaucracy and internal issues were tied for the third biggest bottleneck. Interestingly, the lack of available technology and the complexity of the problem were the last two bottlenecks cited by the respondents.  - From my experience, this is wrong! In all industries, the greatest bottleneck is senior management not recognizing control system cyber security is a critical issue. Fix that as some of the oil/gas and chemical companies have and most of the implementation issues will go away. Also, industrial control system security requires a merging of thought and talent from Operations, IT, and management.  It could be that respondents to the survey provided the answers from their own lens, one of the three areas, and that a merged response would reflect more of a direction that management can control.  I believe we will see more management buy-in and understanding in the coming months, driven by awareness and certainly by regulation. “Types of threats – the report indicates that Malware was 28.8%, Phishing was 18.3%, Data Loss Prevention was 19.4%, Insider was 17.7%, and Crime was 15.7%.” - These appear to be legitimate numbers for IT, but have little relevance to control system cyber incidents. Since when does Phishing affect PLCs or other legacy control system field devices? “Over 50% of the respondents stated that critical infrastructure had already been attacked.” If 50% of North American responders are serious that their critical infrastructure has already been attacked, how can they be doing so little and ignoring so much? Additionally, if these are IT-type attacks, how can the security be so ineffective? The protection of our most critical assets requires understanding and merging of core technologies and corporate groups.  In some ways, the organization chart must all participate and this requires direction from the top.  We are making progress, but not fast enough, and hopefully information in the survey, report and herein will serve as a further catalyst.  The Secure Computing survey results are similar to those conducted by Trusted Network Technologies in 2006 where 50 North American utilities were surveyed and 20% said their SCADA systems had already been compromised and 67% claimed total SCADA awareness. Obviously, something is amiss. There is a need to take swift and careful action, because we really are that unsecure.  There has been enough talk, it is time for executives to act. Joe Weiss

Next week I will be in Washington DC for a Computer Security Institute (CSI) conference session on Wednesday. The session will discuss the needs and issues associated with IT and Operations working together to secure industrial control systems. Ed Goff from Progress Energy will be the “active” moderator as he will also be participating in the discussion.  There will also be an IT security vendor and a control system security vendor on the panel.

While in Washington, I will have a chance to provide a control system cyber security status update to numerous government and congressional representatives.

Joe Weiss

The GE Fanuc/Proficy Information Portal Remote Code Execution Vulnerability has been identified via US CERT Vulnerability Note VU#339345 and issued November 7th as a NERC ES-ISAC Advisory:  “…The NERC ES-ISAC estimates that the risk to grid reliability from this vulnerability is LOW based on the limited deployment of the vulnerable technology…The NERC Advisory contains useful information regarding the affected product. Please forward to technical SMEs within your organization as required to assess and remediate the potential impact of exploit outlined this Advisory… NERC Advisories are not the same as a reliability standard, and your organization will not be subject to penalties for a failure to address this Advisory…”

I had this specific vulnerability demonstrated to me and it was obvious this was not a trivial problem. The GE Fanuc HMI is not widely deployed in electric control centers or substations which are NERC’s traditional venues but is widely deployed in power plants and other industrial facilities. Consequently, it is not clear the risk to grid reliability is low.  In addition, this is not the only GE Fanuc cyber vulnerability.

I did have a chance to discuss this and other disclosure issues with Mike Assante, NERC VP and Chief Security Officer. Among other issues, Mike is in the process of restructuring how NERC issues vulnerability notices.  I believe the new process can help. As mentioned, NERC Advisories are not always treated as critical activities. This was vividly demonstrated with the Aurora and Boreas Advisories that have been pretty much ignored by industry. The GE Fanuc case is even more tenuous as the Advisory designates the vulnerability as a low risk. Will the utilities begin to take these advisories seriously or is more regulation needed?  Without meaning to sound like a broken record, this another example of the need for a CERT for Control Systems.

Joe Weiss

I wanted to address an issue that causes great confusion – what is cyber?  Cyber is not just a 12-year pimply-faced hacker sitting in front of a computer drinking Dr. Pepper and writing malware. Moreover, cyber does not have to be an intentional attack. According to NIST, a cyber incident is an occurrence that actually or potentially jeopardizes the Confidentiality, Integrity, or Availability (CIA) of an information system or the information the system processes, stores, or transmits or that constitutes a violation or imminent threat of violation of security policies, security procedures, or acceptable use policies. Incidents may be intentional or unintentional. (FIPS PUB 200, Minimum Security Requirements for Federal Information and Information System, March 2006.) What is important about this definition is that a cyber incident can be intentional or unintentional, an actual or potential compromise of CIA, or a violation or imminent threat to CIA.  To date, most control system cyber incidents have been unintentional. However, these unintentional incidents have shut down industrial facilities (including nuclear plants), caused significant equipment damage, and even killed people. As mentioned in previous blogs, cyber incidents are not just exploits of traditional IT vulnerabilities such as buffer overflows. Cyber incidents also occur at, and between, devices and systems because of how they are connected. Consequently, cyber is a reliability issue, not just a security issue, and needs to be addressed accordingly. What we need is a new definition to describe impacts on electronic communications between systems whether they be intentional or unintentional. Joe Weiss

I announced a while back that we would be tearing up the editorial calendar and making the November issue a special issue with a single theme: critical infrastructure protection.

I thought I'd share with you the cover, and some of the table of contents...just to whet your appetites, see?

The issue goes to the printer tomorrow, and will be online in a couple of weeks.

Here's a look at some of the stories inside:

My editiorial, "What Is to be Done?" asks who is going to step up and create a Global Critical Infrastructure Emergency Response Team.

Ranjan Acharya, a system integrator from New Zealand confesses to questions about ASCI and ISA99.

Bela Liptak's Lessons Learned column is about Nuclear Plant Security and Cyberterrorism.

John Rezabek's On the Bus column is an end-user's look at cyber.

In Control Report, Executive Editor Jim Montague talks about the stewpot of standards, and how to get in early before all the meat is gone.

In the "feature well," Managing Editor Nancy Bartels steps off with an introduction to "The Security Lifestyle." Then Joe Weiss and I talk reality about cybersecurity-- and how to fix the problem. Next, Jim Montague talks about how security works, in "Carving Up Security." Rich Merritt follows with two articles about physical security, one for outside the plant and one for inside. We nicknamed them "inside" and "outside" when we were working...but we didn't get inside out. Dan Hebert takes on the security of integrated safety systems in his Technically Speaking column, McMillan and Wiener interview DeltaV creator Mark Nixon on safety and security, and Keith Larson talks about the nitty gritty-- tools to make your networks secure.

I'm very pleased with the issue, and I hope you all will be, too.

The Control magazine digital edition is posted. You can read it in Flash format online, and you can also download a DRM-free PDF version and read it at your leisure, or print stories out and pass them around. We welcome you to do that.

http://control.dgtlpub.com/2008/2008-10-31/home.php

This past week was ISA Expo in Houston. As this was the first time in many years I did not attend, I went to Dale Peterson’s blog for his thoughts. He mentioned there was very little Linux on the show room floor. This is not surprising to me. Linux is making headway into the electric T&D world which is not a focus of ISA.

I believe one long term solution to control system cyber security starts with our colleges as they educate the students that will join the end-users, vendors, consultants, and regulators. This includes undergraduate and graduate courses. Because of Professor Ray Vaughn’s previous association with NSA, the Mississippi State computer security program is addressing relevant issues in control systems. Consequently, I was not at ISA but at Mississippi State to give two lectures. One lecture was to a computer science class (I was impressed with the knowledge and interest from many of the students) and the other was open to the University at large as part of Cyber Security Awareness Week. Several items of interest:

-        There is still a hole in the universities for teaching control system cyber security. It requires an interdisciplinary approach and needs to address both policy and technology. Livermore National Laboratory has been charged with developing curricula. I have not seen the results.

-        The university-wide lecture turned out to be a microcosm of the IT and Operations disconnect. Of the more than 100 attendees, only three acknowledged being from engineering - 2 from electrical and one from aerospace. The rest were from the IT community. My feeling is that if the lecture was sponsored by the engineering community, the numbers would probably have been reversed.

-        The Computer Science Department found a significant vulnerability in a control system vendor product (another reason I was impressed with the students). As in many other cases, the vendor has been reluctant to address the vulnerability.  Other discussions from industry have demonstrated similar cyber weaknesses in control systems with similar vendor reticence to address the vulnerabilities. It reinforces the need for an appropriate organization for disclosure – a CERT for Control Systems.

 There was an October 16th report of a study (http://govtsecurity.com/news/ceo-infrastructure-security-1016/) from the National Infrastructure Advisory Council (NIAC). It stated that the government has made great strides working with private industry to secure the nation's critical infrastructures. Another statement followed saying that top executives in the private sector need to step up and do more. I believe there have been strides made, but I question “great strides”.  There are still too many fundamental issues with the cyber security of critical infrastructures such as electric power and water (see multiple previous blogs and congressional testimony). I wholeheartedly agree that CEOs need to be more involved.