Scenario / Questions

The provided 18.04 AMI comes with one interface, and I am adding a second one with its own IP (for stuntman-server). However the instance is only responding to one elastic IP or the other, not both (for SSH and stuntman), while both private addresses do work. Setup with Amazon Linux was also flawless, so should just be a piece of networking config.

There are two network interfaces, each with an Elastic IP and with different subnets on one VPC in one availability zone (172.31.16.0/20 and 172.31.48.0/20) and the same security group.

Ubuntu did not automatically detect the second, but /etc/netplan/50-cloud-init.yaml looks like the place to start from 18.04?

Different from what I am used to and I couldn’t find instructions so guessing this is the problem. I just added an eth1 DHCP to it. And I am guessing these will break if I make an AMI or change the interface (will get new MAC address)?

network:
    version: 2
    ethernets:
        eth0:
            dhcp4: true
            match:
                macaddress: 06:05:14:4a:26:ce
            set-name: eth0
        eth1:
            dhcp4: true
            match:
                macaddress: 06:86:ef:73:d4:1a
            set-name: eth1

This does show up in ifconfig, and I can SSH to both private addresses, but only the second (eth1) responds to either pogram over the EIP, although it does show a few packets on eth0.

ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
        inet 172.31.24.184  netmask 255.255.240.0  broadcast 172.31.31.255
        inet6 fe80::405:14ff:fe4a:26ce  prefixlen 64  scopeid 0x20<link>
        ether 06:05:14:4a:26:ce  txqueuelen 1000  (Ethernet)
        RX packets 413  bytes 48193 (48.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 545  bytes 62679 (62.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
        inet 172.31.52.86  netmask 255.255.240.0  broadcast 172.31.63.255
        inet6 fe80::486:efff:fe73:d41a  prefixlen 64  scopeid 0x20<link>
        ether 06:86:ef:73:d4:1a  txqueuelen 1000  (Ethernet)
        RX packets 899  bytes 403206 (403.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1218  bytes 232644 (232.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 275  bytes 21497 (21.4 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 275  bytes 21497 (21.4 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Find below all possible solutions or suggestions for the above questions..

Suggestion: 1

Likely this is a routing issue, and needs additional configuration entries in the routing tables.

See the following example for instructions how to setup two independent routing tables:

https://serverfault.com/a/823061/476056

In case your application binds to 0.0.0.0 it is required to add rules like the following. Otherwise the outbound traffic will use the default interface, ending up on the first interface.

ip rule add from 0.0.0.0/32 to 192.168.122.0/24 dev eth0 table admin

ip rule add from 0.0.0.0/32 to 192.168.123.0/24 dev eth1 table users

This is not Ubuntu specific, the same occures on CentOS and other distributions.