Through the past few months, more and more ways of providing remote access surfaced within organizations as people were forced to work from home because of the Covid-19 pandemic. This was also the case for remote access to organization critical environments such as industrial control systems in various sectors.
By now and because of the different media stories concerning remote access breaches it should be clear to everybody that you should not just open Remote desktop or any other remote access port to the whole Internet to allow remote access. Instead, you should use at least a secure remote access solution.
Before we dig into the different technical aspects it is important to have a look at the different use cases first because each scenario has its own specific security requirements. These use cases that can or will use a way of remote access are the following (if you think there are missing use cases, please let me know):
- Employees needing access to IT environments only
- Employees needing access to IT and OT environments
- Solution vendors, suppliers or integrator staff
- Managed service providers
The first 3 types often have road worrier type access and are potentially able to connect from everywhere, using an organization provided system to do so. Solution suppliers, vendors, integrators or managed service providers often have either site-to-site VPN tunnels, “phone home” style reverse tunnels or proprietary VPN tunnels that they manage and control themselves.
In Secure remote management for ICS I wrote about some technicalities on that specific topic some years ago. In my blog as well as within various other blog posts on the topic, several good practices have been mentioned and discussed:
- Only use company provided systems for remote access
- Route all traffic through your VPN gateway for better control
- Use a centralized solution
- Use time based solutions – limiting allowed time for access to critical environments
- Use source IP controlled access – more for site-to-site solutions
- Use strong user authentication
- Apply monitoring and logging
However, as I’ve mentioned in Hidden dangers of remote management you can not (always) control who is sitting behind the system.
This makes the use of jump servers a mandatory fact to have secure remote access environment. Security of the jump server(s) itself is not to be neglected and the following elements should be considered when deploying such system:
- All actions should be logged and monitored on the jump server itself as well as centrally
- Do not mix vendors – use separate jump servers for different vendors as you do not want any remotely accessing party to access systems from other parties
- Use separate channels/systems for file transfer and keyboard/video/mouse traffic
- Enforce malware scanning on all transferred files
- Do not use one of the ICS systems as jump server, use a dedicated system within a DMZ for that purpose
- Put the jump server(s) in separate network segments, do not mix with other systems
- Use host firewalls on jump servers to prevent jump servers from connecting to or through each other
Regardless the use case, an aspect very important to invest time in, is network segmentation/segregation of the environment, limiting what can be done on your networks. Such network zoning strategy should include an access matrix, describing all allowed traffic flows and what other mitigating measures are required.
To provide secure access to the ICS environment, direction connections from where the VPN originates all the way down to a VPN gateway as close as possible to the ICS/OT environment to remotely connect to is not such a good idea and should be avoided and prevented at all times. The reason for avoiding this setup is that you do not have control over the traffic passing through that VPN connection except for at the terminating VPN gateway probably – if that device supports traffic inspection.
When using jump stations, the vpn gateway can be placed close(r) to the jump station itself to avoid traffic disclosure/alteration when passing through your IT environment, but then traffic originating from the jump station must be filtered and controlled properly. Never allow direct traffic between jump stations and field devices / PLC’s for example.
Depending on the purpose of the remote access, extra user authentication on a OT firewall might be required. I would even go that far to make this a recommended approach. You might even go that far as using a 2nd layer of VPN if necessary requiring you to have a tunnel within a tunnel to access the critical environments. That additional tunnel can be configured more restrictively than the “regular” first IT VPN access for example.
Another element within network segmentation is that a lot of VPN setups are still using an any-to-any rule within the VPN tunnel. Avoiding such any-to-any rule can be a quick win to limit remote access possibilities. This should preferably be combined with a jump station to be more secure.
Certificate based authentication
When using certificate based authentication (system certificates for managed devices or user certificates for mobile email access) there is another element often forgotten or even neglected. This is the management of certificates within the PKI environment.This means:
- Revoke expired certificates
- Make sure the certificate revocation list (CRL) is published to a reachable location (for the VPN devices and for connecting devices)
- Make sure certificates of decommissioned system are immediately revoked
- Revoke user certificates when people leave the company
- Make sure the VPN gateway used is actually verifying the CRL to start with
Vendor (reverse) tunnels…
For remote access for suppliers, integrators, vendors etc. we can define 2 different types:
- The “real” site-to-site VPN tunnels – usage of a dedicated jump server and all other security related items as mentioned before should be taken into account.
- The reverse tunnels through outbound connections initiated from the to-be managed OT system(s)
Several vendors use a “phone home” solution for secure remote access. Yet this brings some issues or difficulties with it on security level. Yes, the traffic is end-to-end encrypted (although you can not be sure about the vendor side). The possible ways of somewhat controlling this type of remote access are:
- limiting the target IP address the solution if connecting to – yet this requires Internet access for a potential critical system
- Requiring the phone-home solution to be initiated from a DMZ system only (jump server setup) – so that you have better control over the connections made to the rest of your environment
One of the big questions to ask with providing a site-to-site VPN tunnel is: How good do you trust the vendor / integrator / solution providers network?
VPN device flaws…
Last but not least, you have to realize that whatever solution you use for providing secure remote access, be aware that VPN devices also can have vulnerabilities…