Have you ever tried to create a Security Group with a Source or Destination rule that references another Security Group? how about referencing a Security Group from another AWS account to allow ingress network traffic over a Transit Gateway architecture? if this question peaked your interest then you should keep reading.

In this blog we will go into:

Prerequisites
What we like to have
What we probably end up doing most of the time
What we could do instead using AWS Customer-managed Prefix Lists
Considerations

Prerequisites

This blog builds on top of the Prefix List patterns I described in this blog: AWS Prefix List, so have a read of it to provide you with a better context as you read on.

What we like to have

How many of us have tried to implement the following architecture but realised it was not technically possible?

I myself have certainly tried to implement this a couple of years ago but to no avail; recently, a client said they also tried to implement this very same pattern, as per usual I did a bit of googling and confirmed that it is still the case today.

What we probably end up doing most of the time

This is probably what most of us do to allow cross-account network traffic to ingress into an EC2 instance over a Transit Gateway architecture.

In VPC A, instead of being able to reference a Security Group (outside of AWS account A, so from either account B, C or D) as the Source traffic of an ENI (via Security Group rules) attached to the EC2 instance in VPC A, one of the current methods is to add the CIDR blocks of the source traffic in the Source rules in the Security Group in VPC A: the CIDR value could either be the entire VPC CIDR block (of VPC B, C or D) to allow all traffic from a VPC, or, a Subnet's CIDR block to narrow down the ingress traffic to flow only from within a sub-section of a source VPC; or, the specific Private IP addresses of the source EC2 instances (e.g. 172.20.15.1/32).

The approach you decide for this pattern depends on the level of security posture you are comfortable with implementing into your network architecture:

VPC CIDR block values: this will allow ingress traffic wide open from the entire source externally VPC, if you intend for all resources from a source VPC to send traffic to your target resources then this option is fine
Subnet CIDR block values: this provides a narrower approach with a slightly tighter level of network security than above, if you intend for all resources from a source Subnet to send traffic to your target resources then this option is fine
Specific CIDR values of a Private IP addresses: this option provides the tightest network security control of the 3 options, however, maintaining a list Private IP addresses of EC2 instances outside of your AWS account (whether you or a 3rd party owns the account) will require a some operational effort. The solution proposed below will provide an automated mechanism to solve this particular problem

Network security controls could be further tightened when coupled with the use of NACLs, have a read of my blog for an example of incoroporating NACLs into your network architecture: Swiss Cheese Network Security: Factorising Security Group Rules into NACLs and Security Group Rules

An example scenario that could be problematic for this architecure is that, if the Source Private IP addresses (for resources outside of the account) needs to be constantly added or removed in the Security Group in VPC A - pet EC2 instances being provisioned and terminated: this will be a burden for the operations team as they would constantly need to update the Security Group rules to relfect changes happening outside of the AWS account - this would not be a problem if we were able to reference in rules the Security Groups from other AWS accounts, perhaps one day AWS will have this ability. This is especially burdensome when you have to co-ordinate changes with 3rd party owners of the AWS accounts outside of your control, imagine having to maintain changes from a dozen external AWS accounts.

What we could do instead using AWS Customer-managed Prefix Lists

Here we propose a pattern to achieve the same outcome but instead we leverage Prefix Lists, to externalise the management of CIDR blocks in the AWS accounts (B, C and D) where the network traffic originate from, then reference the external Prefix List in each of the accounts (B, C, and D) in the Security Group rules of account A; with the help of AWS Resource Access Manager (RAM) as Prefix Lists as shared with AWS account A by account B, C and D.

In the diagram above we have 3 options for the CIDR values maintained in these Prefix Lists outside of AWS account A, these types of values are similiar to the 3 options when the rules were defined (explained earlier) in the Security Group in VPC A, but the principle of network security controls remains the same in terms of tightness.

This pattern achieves the same outcome as what we desire if Security Groups could be (it is not supported by AWS at the time of writing this blog) referenced over a Transit Gateway, but it does have its drawbacks: the Max Entries (not the actual) of a Prefix List is counted towards the Quota of the Security Group that references it – so the example illustrated above results in 3 rules (1 for each Prefix List for each account) created in the Security Group in VPC A. This patterns has merits when you want allow inversion of control to enable external AWS accounts to control what network traffic is allowed to enter with the help of using Prefix Lists shared through RAM - remember the control is essentially delegated to the external AWS accounts, so you have to trust the level of scoping for CIDR value entries is being maintained in these accounts.

Bonus - Extra tight network security controls

The pattern above solves a small to medium sized problem on its own, but if we were to combine it with the patterns detailed in these two blogs: Leveraging AWS Prefix Lists and Maintaining a Prefix List of EC2 Private IP Adresses using AWS EventBridge, we can achieve the following:

By combining the 3 patterns we will end up with a network architecture that achieves the following:

A work-around for cross-account Security Group reference over a Transit Gateway.
List of Private IP addresses of similar EC2 instances (any grouping of your choosing) automatically tracked and managed in a Prefix List within each spoke account based on a Tagged value on the EC2 instances.
The same Prefix List in each spoke account can be referenced (via Resource Access Manager) to route return traffic back from the Subnet Route Table in VPC A to the originating Transit Gateway – this could potentially fully automate routing of traffic to Transit Gateway – great for scenarios where you only want return traffic for one or two IP addresses (especially when they are pets) in account B, C or D.
The same Prefix List in each spoke account can be referenced (via Resource Access Manager) to route return traffic back from Transit Gateway to the destined source Transit Gateway Attachment – this could potentially be used to automate routing if static or propagated routing is not used in a Transit Gateway Route Table. We can narrow it down to a very small subset of distributed allowable return traffic IP block for a spoke source traffic attachment – so only a subset of return traffic is allowed to return back to the originating TGW spoke.
Depending on the narrowness of the CIDR values used in the Prefix Lists, e.g. a few distributed /32 IPs in the Prefix List for a source VPC with a 1024 addresses for it's CIDR block, if used effectively, least privilege for network security is achieved using this pattern.

Considerations

As with any patterns, services or components, the pros and cons of each one needs to be weighed against each other and thought out in the interest of the long-term overall benefits for your solution and most importantly for your organisation. Restructuring existing networking and migrating workloads into it can be difficult and time consuming - especially if manual steps to deploy your infrastructure is required. Use Prefix Lists economically so that you do not under consume the number of Max Entries set by leveraging Lambdas to automatically update Max Entries; check out my blog on Maintaining a Prefix List of EC2 Private IP Adresses using AWS EventBridge.

This solution compliments the use of networking solutions in other blogs I have written:

Work-around for cross-account Transit Gateway Security Group Reference