Why setting the http_proxy and https_proxy environment variables has no effect on Docker for Windows (or, why there is no docker.ini or Windows Registry setting for this)

Docker for Windows allows you to use docker commands from cmd or PowerShell as if the Docker host were running locally on your PC. This is a clever illusion: while docker.exe is a Windows program, the Docker host you’re connecting to is really running in a Hyper-V virtual machine named MobyLinuxVM. So far, I have not found a way to SSH into this VM; had I done so, I could have set the HTTP_PROXY and HTTPS_PROXY environment variables by PowerShell script every time my PC’s IP address changed.

Caveats before beginning

I use this setup on Windows 10 and Docker for Windows 17.06. A colleague still using Docker for Windows 17.03 reports that he has over 20 instances of the DummyDesperatePoitras virtual switch, so if you’re not on 17.06 yet, I recommend updating.

The IP address (169.254.123.45), DNS host (10.1.1.1) and domain (mandie.net) are DEFINITELY not the right options for your configuration, and are just here to make the examples easier to read. Do NOT just cut and paste things from this page – but you shouldn’t do that anyway 😉

DummyDesperatePoitras: this weird Docker for Windows artifact turns out to be key

The Hyper-V DummyDesperatePoitras virtual switch gets made when the Docker virtual switch is. This is a workaround for some issue or other. It turns out to be extremely useful! I’d tried using the Docker virtual switch’s IP address as the proxy address, but that didn’t work.

DummyDesperatePoitras gets a random, non-routable (169.254.0.0/16) IP address.

Get that IP address using PowerShell (no need to be admin):

(GetNetIPAddress -InterfaceAlias "*DummyDesperate*" -AddressFamily IPv4).IPAddress

For this example, I’ll use 169.254.123.45 as the result of this command. Your randomly-generated IP address will almost certainly be different.

If you get more than one result, it might be due to the problem described earlier in this post that some older versions of Docker for Windows had, where the Docker virtual switch was removed every time and remade, without removing the DummyDesperatePoitras, but still being made every time the Docker virtual switch was made. As mentioned earlier, try uninstalling Docker for Windows (removing the MobyLinuxVM will remove any images or containers you’ve created), remove all the DummyDesperatePoitras virtual switches, and then installing the 17.06 or later version.

Just enough CNTLM

CNTLM is an executable primarily available on SourceForge (now under new management and no longer dispensing spyware with its downloads, but still…) that has not been updated in over 5 years. It runs as a service, which requires local admin. It goes against all my principles.

It is also the only practical way to use Unix-style command line tools with NTLM-authenticating corporate web proxies.

Promising alternative: the Python-based Windows proxy px (https://github.com/genotrance/px). It has the major advantage of not requiring your Windows password in any form, and the major disadvantage of needing to understand more than I do about Python to build and run it. If you manage this feat, please tell me about it, and better yet, write your own blog post about how you pulled it off.

Listening on 127.0.0.1:3128 or just 3128 and setting http://127.0.0.1:3128 as your proxy will not work for Docker for Windows, because the guest VM that is really your Docker host interprets that address as *itself*, not your Windows host running CNTLM.

Having it listen to 0.0.0.0:3128 and setting the PC’s IP address as the proxy works, but you have to then change your Docker settings any time your PC gets a new IP address – a routine occurrence for consultants or anyone moving around a large campus. It also means that your PC will act as a web proxy for ANYTHING that isn’t blocked by its firewall. This can be either a bug or a feature, but I officially recommend that you do not use it as a proxy for things outside of your desktop.

The advantage of having CNTLM listen on a non-routable but static IP address is that only traffic that originated on your PC can use the proxy, but you still don’t have to change it when your PC’s IP address changes.

CNTLM configuration is set in cntlm.ini, under C:\Program Files (x86)\cntlm. The easy way to use it is to set your Windows password directly in the file. That is also the terrible way to use it. If you store the NTLMv2 hash, at least it can’t be used anywhere other than your PC (which is still not great). Set your cntlm.ini file to only be readable to the account the cntlm Windows service is running under.

Here is a way to get your NTLMv2 hash: https://stackoverflow.com/a/44238035

Set CNTLM to listen for (DummyDesperatePoitras IP) 3128 – in this example:

Listen 169.254.123.45 3128

Remember to set your NoProxy subnets here: usually, 127.0.0.*, 10.*, 172.16.*-172.31.* and 192.168.*, but you might have others that shouldn’t go via the corporate web proxy.

Set your http_proxy and https_proxy in Windows for other command line tools

While testing, set it at the command line for your current session – note the http:// (not https://) for both.

For cmd:

set http_proxy=http://169.254.123.45:3128
set https_proxy=http://169.254.123.45:3128

PowerShell:

$env:http_proxy = "http://169.254.123.45:3128"
$env:https_proxy = "http://169.254.123.45:3128"

This will only have effect for this session, and only as long as you have the window open.

Once you’re sure this works, set them in your profile’s environment variables. This change will not take effect until you log off and log back on to Windows.

Docker for Windows settings: Proxies and daemon.json

Right-click the little whale icon in the system tray and select “Settings…”, then click “Proxies”

Docker Proxy Settings

In the “Web Server (HTTP)” blank, type http://169.254.123.45:3128 (replace with the IP address you got from running the Get-NetIPAddress cmdlet at the beginning) and click the “Use same for both” box. You should set your proxy bypass addresses in cntlm.ini for consistency between all the command line tools you use.

Click “Apply” – this is necessary to save your changes (writing them to MobyLinuxVM), but will also restart Docker (MobyLinuxVM). Go get a coffee – this takes a minute or two.

Next, click “Daemon” so that we can give it a DNS server to use for your internal domains. Click the “Experimental features” checkbox and the “Basic” switch so that it turns to “Advanced”

Docker Daemon – Enable daemon.json editing

Add entries for dns and dns-search, remembering to separate each entry with a comma.

Your resulting JSON file should look like this, but with your internal domain and DNS server’s IP address instead of mandie.net’s:

{
 "registry-mirrors": [],
 "insecure-registries": [],
 "debug": true,
 "experimental": true,
 "dns": [
 "10.1.1.10"
 ],
 "dns-search": [
 "mandie.net"
 ]
}

Click “Apply,” and wait that minute or two again.

Testing, and extra help for docker build

Were we successful? Open cmd or PowerShell, and try pulling the hello-world image. Even if you have the latest version already, this goes out to docker.io and checks.

docker pull hello-world:latest

This configuration should be sufficient for pulling Docker images and running containers on your Docker host. However, any image builds that pull components from outside (apt-get, npm, etc.) will need to get the proxy information explicitly. You don’t want to store this in your Dockerfiles, since it’s specific to your PC. You can feed build-time environment variables like this:

docker build --build-arg http_proxy=http://169.254.123.45:3128 --build-arg https_proxy=http://169.254.123.45:3128 -t myawesomeimage:latest .

Let me know if something about this doesn’t work for you – I’m still a bit surprised that it works at all.

In case you were wondering why I’ve not posted any silly Skype tricks lately, I seem to have ended up in charge of Kubernetes at my company, and for reasons I won’t bore you with, we’ve decided to set up IBM Cloud private on-prem.

The product certificate expired on 22 August 2017, so there’s a patch: https://www.ibm.com/developerworks/community/blogs/fe25b4ef-ea6a-4d86-a629-6f87ccf4649e/entry/Certificate_update?lang=en

However, the included bash scripts presume that all of the nodes have access to Docker Hub. Since ours don’t, because we are not heathens who allow our servers to talk to the Internet, I made a workaround:

1) Get the updated ibmcom/cfc-router:1.2.0 image using a machine that can connect to Docker Hub and pack it into a tarball:

docker pull ibmcom/cfc-router:1.2.0
docker save ibmcom/cfc-router:1.2.0 -o cfc-router-fix.tar

(rant about pushing an updated Docker image with THE SAME EXACT VERSION TAG goes here)

2) Copy this tarball to all of the nodes in your cluster

3) SSH into each node in the cluster and load the image:

docker load -i /wherever/you/put/it/cfc-router-fix.tar

4) Copy the update-cert.sh script and remove the following lines, because they are the ones that wanted to go onto the scary Internet and pull the images directly. Leave in all the other docker commands! If you’re using Windows, copy it to a Linux machine and edit it there to avoid any weird linefeed stuff:

if [[ "$(uname -m)" == "x86_64" ]]; then
    docker pull ibmcom/cfc-router:1.2.0
else
    docker pull ppc64le/cfc-router:1.2.0
fi

5) Copy your modified script to each of the nodes. You then need to set execute permission on it, and can finally run it:

chmod +x update-certs.sh
./update-certs.sh

How your AudioCodes SBA login pages should look, as of May 2016

Unlike the components the SBAs share with their big Front End Server brothers, like RTCSRV and RTCMEDSRV, the manufacturer-custom web interfaces are NOT updated in the Cumulative Updates. You need to check on these periodically with AudioCodes, Sonus or whoever else you got your SBAs from.

These management interface updates are for security and performance issues. If you’re running your SBAs the way the manufacturer recommended, though, there are a lot of remote operations that just won’t work, making version checking painful.

However, if you’ve got AudioCodes SBAs, here is a one-liner that only requires that you have a consistent naming convention (we have “sba” in all of our SBA names) and at least ViewOnlyAdmin access to Lync/Skype, using the magic of very simple webscraping:

(Get-CsPool).computers.where({$_ -like "*sba*"}) | foreach { (Invoke-WebRequest -Uri "http://$_/Home/LogOn").content -match "(1\.\d+\.\d+\.\d+)" | out-null; [pscustomobject]@{ComputerName = $_; Version = $matches[0] } }

Substitute whatever your SBAs have in common for “*sba*” (remember the asterisks!)

Or, here’s a version using nested Where() expressions that will work even if you have no naming conventions:

(Get-CsPool).where({$_.services -like "*registrar*" -and $_.services.where({$_ -like "WebServer*"}).count -eq 0}).computers | foreach { (Invoke-WebRequest -Uri "http://$_/Home/LogOn").content -match "(1\.\d+\.\d+\.\d+)" | out-null; [pscustomobject]@{ComputerName = $_; Version = $matches[0] } }

I have no idea if a similar approach will work with other manufacturers’ SBAs.