Secure Kubernetes Networking: Default-Deny with Per-Project Egress Allowlists via Envoy Proxy

Modern Kubernetes clusters must balance security and operational agility. Default-deny NetworkPolicy is a critical security control, but enforcing it at scale—especially for dynamic agent-based workloads—requires careful design. This guide explores how to implement a zero-trust network model using Kubernetes NetworkPolicy with default-deny rules and per-project egress allowlists, leveraging Envoy proxy for granular control over agent execution.

Why Default-Deny NetworkPolicy?

Default-deny policies enforce least-privilege networking by blocking all traffic unless explicitly allowed. This reduces attack surfaces, limits lateral movement, and aligns with zero-trust principles. However, default-deny can introduce friction for legitimate workloads, particularly when:

• Agents (e.g., CI/CD runners, monitoring probes, or custom workloads) need outbound connectivity to external services.
• Projects require isolated egress paths to specific APIs or internal services.
• Dynamic workloads (e.g., ephemeral agents) must avoid manual policy updates.

Challenges with Default-Deny in Kubernetes

1. **Policy Complexity**: Managing per-project allowlists manually scales poorly.
2. **Agent Flexibility**: Agents often need dynamic egress rules based on their purpose (e.g., a CI agent may need GitHub access, while a monitoring agent needs Prometheus endpoints).
3. **Envoy Integration**: Envoy proxies can act as a layer-7 filter, but integrating them with Kubernetes NetworkPolicy requires careful design.

Solution Architecture

This approach combines:

• Kubernetes NetworkPolicy: Default-deny ingress/egress at the pod level.
• Envoy Proxy Sidecars: Per-pod Envoy instances to enforce fine-grained egress rules.
• Project-Based Allowlists: Centralized configuration for each project’s allowed destinations.
• Dynamic Rule Injection: Envoy filters populated at runtime via Kubernetes ConfigMaps or Secrets.

Step 1: Default-Deny NetworkPolicy Basics

A default-deny policy blocks all traffic unless explicitly allowed. For egress, this means:

“`yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-egress
namespace: default
spec:
podSelector: {}
policyTypes:
– Egress
egress: [] # Empty list = deny all
“`

Key Considerations

• podSelector: {} applies to all pods in the namespace.
• Ingress policies should also be defined (e.g., allow cluster-internal traffic only).
• Test in a staging environment first—default-deny can break workloads.

Step 2: Envoy Proxy for Egress Control

Envoy sidecars can filter egress traffic based on:

• Destination IP/hostname.
• HTTP headers (e.g., `Authorization`).
• Ports and protocols.

Implementation Steps

1. **Deploy Envoy Sidecar**: Use an init container or admission webhook to inject Envoy into pods needing egress control.

Example Deployment with Envoy:
“`yaml
spec:
containers:
– name: app
image: my-app
– name: envoy
image: envoyproxy/envoy:v1.25-latest
ports:
– containerPort: 15001 # Envoy admin port
volumeMounts:
– name: envoy-config
mountPath: /etc/envoy
volumes:
– name: envoy-config
configMap:
name: envoy-config
“`

2. **Configure Envoy Filtering**: Define allowlists in a ConfigMap. Example for a CI agent:
“`yaml
# envoy-config.yaml
static_resources:
listeners:
– name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 10000
filter_chains:
– filters:
– name: envoy.filters.network.http_connection_manager
typed_config:
“@type”: type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
route_config:
name: local_route
virtual_hosts:
– name: local_service
domains: “*”
routes:
– match:
prefix: “/”
route:
cluster: external_service
timeout: 0s
# Allowlist for CI agent
allowlist:
– “api.github.com:443”
– “registry.example.com:443”
“`

3. **Dynamic Rule Injection**: Use Kubernetes Secrets or ConfigMaps to update allowlists without pod restarts. Example:
“`yaml
# Allowlist for project X
apiVersion: v1
kind: ConfigMap
metadata:
name: project-x-allowlist
data:
allowed_domains: “api.example.com,metrics.example.com”
“`

Step 3: Per-Project Egress Allowlists

To avoid manual policy updates, centralize allowlists by project. Example structure:

“`yaml
# Allowlist per project (e.g., in a GitOps repo)
apiVersion: v1
kind: ConfigMap
metadata:
name: project-allowlists
namespace: gitops
data:
project-ci:
“api.github.com:443”
“registry.example.com:443”
project-monitoring:
“prometheus.example.com:9090”
“grafana.example.com:3000”
“`

Automating Allowlist Injection

Use a Kubernetes admission controller or sidecar to inject Envoy filters dynamically:

1. **Admission Webhook**: Validates and injects allowlists during pod creation.
2. **Sidecar Init Container**: Pulls the allowlist ConfigMap and updates Envoy config.

Example init container:
“`yaml
initContainers:
– name: envoy-configurer
image: curlimages/curl
command: [“/bin/sh”, “-c”]
args:
– curl -s https://raw.githubusercontent.com/your-repo/allowlists/main/project-$(POD_NAMESPACE).yaml > /etc/envoy/allowlist.yaml;
envoy –config-path /etc/envoy/envoy.yaml –bootstrap-config /etc/envoy/allowlist.yaml
“`

Step 4: Testing and Validation

1. **Test Default-Deny**: Deploy a pod with no allowlist and verify it cannot reach external services.
2. **Validate Allowlists**: Ensure pods with allowlists can reach allowed destinations but are blocked from others.
3. **Monitor Envoy Logs**: Use Envoy’s admin interface (`http://:15000`) to inspect filtered traffic.

Step 5: Scaling and Maintenance

1. **GitOps for Allowlists**: Store allowlists in a Git repository and sync them via ArgoCD or Flux.
2. **Automated Compliance Checks**: Use tools like OPA/Gatekeeper to enforce allowlist policies.
3. **Audit Logging**: Export Envoy access logs to a central system for forensic analysis.

Common Pitfalls and Mitigations

| Pitfall | Mitigation |
|———————————-|—————————————————————————–|
| Envoy config errors | Validate configs with `envoy –validate-config /etc/envoy/envoy.yaml` |
| Allowlist drift | Use GitOps to sync allowlists with source truth |
| Performance overhead | Tune Envoy stats and use connection pooling |
| Agent misconfiguration | Test allowlists in a staging cluster before production |

When to Consider X-ITM Platform Engineering

Implementing this pattern at scale requires:

• Expertise in Kubernetes networking, Envoy, and GitOps.
• Automated validation of allowlists and policies.
• Performance tuning for high-throughput clusters.
• Integration with existing CI/CD and monitoring stacks.

If your team lacks the bandwidth to design, test, and maintain this architecture—or if you’re evaluating whether default-deny with Envoy is the right approach for your workloads—X-ITM Platform Engineering can help. Our team has helped organizations like yours implement zero-trust networking at scale, balancing security with developer productivity. Whether you need a bounded assessment to validate the approach, a pilot to test Envoy integration, or ongoing platform engineering to maintain and evolve your network policies, we provide the senior engineering expertise to get it right.