Issue: sudo service docker start says "Starting Docker: docker" but sudo service docker status shows "Docker is not running" and most docker commands throw following message "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?".

abhijit@AwsJunkie:~$ sudo service docker start
 * Starting Docker: docker                                    [ OK ]
abhijit@AwsJunkie:~$ sudo service docker status
 * Docker is not running
abhijit@AwsJunkie:~$ docker version
Client: Docker Engine - Community
 Version:           23.0.0
 API version:       1.42
 Go version:        go1.19.5
 Git commit:        e92dd87
 Built:             Wed Feb  1 17:47:51 2023
 OS/Arch:           linux/amd64
 Context:           default
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Open the docker log  /var/log/docker.log to check the error message.

abhijit@AwsJunkie:~$ cat /var/log/docker.log
time="2023-02-03T22:40:10.760588900-05:00" level=info msg="Starting up"
time="2023-02-03T22:40:10.763542200-05:00" level=info msg="libcontainerd: started new containerd process" pid=5914
time="2023-02-03T22:40:10.763667700-05:00" level=info msg="[core] [Channel #1] Channel created" module=grpc
time="2023-02-03T22:40:10.763682100-05:00" level=info msg="[core] [Channel #1] original dial target is: \"unix:///var/run/docker/containerd/containerd.sock\"" module=grpc
time="2023-02-03T22:40:10.763704000-05:00" level=info msg="[core] [Channel #1] parsed dial target is: {Scheme:unix Authority: Endpoint:var/run/docker/containerd/containerd.sock URL:{Scheme:unix Opaque: User: Host: Path:/var/run/docker/containerd/containerd.sock RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}" module=grpc
time="2023-02-03T22:40:10.763713700-05:00" level=info msg="[core] [Channel #1] Channel authority set to \"localhost\"" module=grpc
time="2023-02-03T22:40:10.763928700-05:00" level=info msg="[core] [Channel #1] Resolver state updated: {\n  \"Addresses\": [\n    {\n      \"Addr\": \"/var/run/docker/containerd/containerd.sock\",\n      \"ServerName\": \"\",\n      \"Attributes\": {},\n      \"BalancerAttributes\": null,\n      \"Type\": 0,\n      \"Metadata\": null\n    }\n  ],\n  \"ServiceConfig\": null,\n  \"Attributes\": null\n} (resolver returned new addresses)" module=grpc
:
:
failed to start daemon: Error initializing network controller: error obtaining controller instance: unable to add return rule in DOCKER-ISOLATION-STAGE-1 chain:  (iptables failed: iptables --wait -A DOCKER-ISOLATION-STAGE-1 -j RETURN: iptables v1.8.7 (nf_tables):  RULE_APPEND failed (No such file or directory): rule in chain DOCKER-ISOLATION-STAGE-1
 (exit status 4))

Error message:

failed to start daemon: Error initializing network controller: error obtaining controller instance: unable to add return rule in DOCKER-ISOLATION-STAGE-1 chain: (iptables failed: iptables --wait -A DOCKER-ISOLATION-STAGE-1 -j RETURN: iptables v1.8.7 (nf_tables): RULE_APPEND failed (No such file or directory): rule in chain DOCKER-ISOLATION-STAGE-1
(exit status 4))

Configure iptables with alternative legacy one /usr/sbin/iptables-legacy (selection number: 1)

abhijit@AwsJunkie:~$ sudo update-alternatives --config iptables
There are 2 choices for the alternative iptables (providing /usr/sbin/iptables).

  Selection    Path                       Priority   Status
------------------------------------------------------------
* 0            /usr/sbin/iptables-nft      20        auto mode
  1            /usr/sbin/iptables-legacy   10        manual mode
  2            /usr/sbin/iptables-nft      20        manual mode

Press <enter> to keep the current choice[*], or type selection number: 1
update-alternatives: using /usr/sbin/iptables-legacy to provide /usr/sbin/iptables (iptables) in manual mode

To confirm start docker then check status or run any docker command.

abhijit@AwsJunkie:~$ sudo service docker start
 * Starting Docker: docker                                    [ OK ]
abhijit@AwsJunkie:~$ sudo service docker status
 * Docker is running
 abhijit@AwsJunkie:~$  sudo docker version
Client: Docker Engine - Community
 Version:           23.0.0
 API version:       1.42
 Go version:        go1.19.5
 Git commit:        e92dd87
 Built:             Wed Feb  1 17:47:51 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.0
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.5
  Git commit:       d7573ab
  Built:            Wed Feb  1 17:47:51 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.16
  GitCommit:        31aa4358a36870b21a992d3ad2bef29e1d693bec
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

References