Practical side of FAT/SAT testing

While listening to one of the talks at the ISC-CPH conference back in 2022, I realized that nobody is talking about the practical side of performing cyber security tests in a FAT/SAT testing approach. It is not only performing the FAT/SAT testing itself that is important but all things that come with it and all necessary actions and items to bring said testing to a good and acceptable result. This also means following a certain FAT/SAT testing process in which the preparation and reporting steps might be the most important ones.

To prepare for a test you should…

• Get all possible information => know the context of the thing to be tested
  • Know where the device will reside
  • Know how it will be used by the end user
  • Know what (if any) file transfer will be used
  • Know if there will be remote access or not
  • Know if it is/will be a standalone system or not
• Check cyber security requirements – if these have been set upfront (which they should definitely!)
• Get a work permit prior to start testing: Not only because then the valid change management processes are followed, but also that you are aware of all personal protective equipment that you will need but also to make sure that a last minute risk assessment is done so that you as a tester know what potential hazards there are at the testing location and to make sure that the necessary people know about the tests should something odd be observed during testing. You can consider the work permit being equal as the “Get out of Jail Free cards” often used during testing in IT environments.

Some important elements while performing testing…

• Your tests should follow a standard approved test process – while leaving room for additional tests should this be necessary. This is important for repeatability of your tests.
• Make sure that your tests cover ALL requirements mentioned in the preparation step – you might need to be able to explain how your tests are covering those requirements
• Make sure your tests are not showing “all is clear” (green lights) while in reality your test results are useless because of a lack of information or permissions to be able to perform a scan (Nessus Auth Scanning tool)
• Make sure there is a backup or image of the system you are going to test.
• Don’t use Eicar but use a “dirty USB stick”, loaded with Malware. This will only be possible during security FAT tests…
• Make sure to verify those passwords… especially during SAT as these tend to be “forgotten” to be changed…
• Depending on the “future” physical location of the system under tests,operator Jail breakout testing will be necessary => Presentation operator jail breakout testing
• While performing testing it is important to verify the logical & Physical reachability of your devices.

Reporting

• Document every single step that has been done/changed or used to be able to perform the tests
• Include potential attack paths, combining all vectors (physical, logical and human)

The most important element though is making sure that all industrial processes are followed within the environment you are performing a security test in. In a FAT testing environment, typically at the vendor location, this will probably be less important but still…

Through performing several security tests within industrial installations over the past 20 years, I also assembled an “assessment jump bag” with all possible accessories you can imagine, and yet this is probably not yet enough as sometimes I run into situations where I need some specific thing that I just don’t have yet in the jump bag.

Some items in my jump bag are:

• Camera
• Note books
• All possible USB cables and adapters
• Tweezers
• (long) (wooden) sticks
• “dirty USB sticks” (not dirty the physical state meaning, but dirty as in containing a lot of malware)
• (portable) USB keyboard – with additional USB ports on it
• USB to Ethernet/serial/… adapters
• Screwdrivers
• Umbraco key(s)
• Rittal key(s) => most of the time you don’t need these as those are almost always present in the vicinity of a rack or cabinet…
• Ethernet cables
• Switch port (capable of performing port mirroring and span ports)

How a security testing program could look like: