# -*- mode: text; indent-tabs-mode: nil -*-
#
# Blow-by-blow directions for setting up an XP image from scratch.
# 
# These are raw notes and commands to paste into a shell.
# Mostly Bash shell commands for Windows, some tcsh commands for Boss or Ops.
# Some (most?) of it could be scriptified with some work. 
# 
# A major prerequisite is a Windows XP image that doesn't require activation on each
# installation.  We use the MSDN August, 2001 image, updated by Service Packs and
# Windows Update.  "Bulk license" versions should work as well.

# Notice that this file has spaces instead of tabs at the beginning of lines.  A tab
# in either Bash or tcsh causes it to display all of the possible command completions!

# Here's a little Emacs keyboard macro to ease the copy-and-paste business:
    ; Copy a command line, leaving off the whitespace on the beginning of the line.
    (fset 'copy-command-line [?\M-m ?\C-  ?\C-e ?\C-f C-insert])
    (global-set-key "\^C\^E" 'copy-command-line)

  # By convention, optional "informational" commands are indented a couple of spaces more.
  ## Debugging and problem-solving stuff is double-# commented.

# TCSH aliases used below:
alias v 'ls -lsF'               # "Verbose" listing
setenv en emulab.net
alias rootpc 'sudo ssh pc\!^.$en \!:2*'
alias rootrd 'rdesktop -K -g 1280x1024 -a 16 -u root pc\!^.$en &'

# In Bash,
alias v='ls -lsF'

# To make UNC paths like //fs work in Cygwin below, either:
# . Explore to \\fs\share, giving *your* login name and Windows password,
# . or use the "net use" command to provide it.
  net use Z: '\\fs\share' 'mypwd' /user:mylogin /persistent:no

================================================================
    . Start with a clean XP image, as it comes from the CD.

      - How to do the Windows XP installation and make a WINXP-BASE image:

        . Swap in a firewalled experiment with a FBSD-STD image.

            # Firewall while making Windows images.
            set fw [new Firewall $ns]
            $fw set-type ipfw2-vlan
            $fw set-style basic

        . Do "node_admin on" so PXE throws it into the FBSD MFS.
        . Hook up a console, reboot and modify the boot order to boot from the Windows XP CD.

        . Do the Windows installation, adding whatever drivers are needed.
          - 8 gig is a good size for the XP NTFS partition, of which 4-5 will be user space.
            (They can always allocate more later with Disk Manager.)
          - You'll save some work later if you make the first admin user account named "root".

        . Set the usual password for root in Control Panel / User Accounts.
            It asks whether to make the root files private.  I've been saying "no".
          - While you're there, click "Change the way users log on or off" and make
            sure "Use Fast User Switching" is turned on.

        . Make sure you turn on Remote Desktop logins under Control Panel / System / Remote (!)

        . Go into Control Panel / Power Options / Hibernate, and make sure it's disabled.
          This will give you extra gigs of disk space on C: equal to your RAM size .
          Also set Power Options / Power Schemes to "Always On".

        . Set the workgroup name to EMULAB in Control Panel/System/Computer Name/Change...
          No need to reboot yet if you have more to do.

        . On reboot, restore the boot order so PXE goes back into the FBSD MFS, and make an image.
          I called mine WINXP-BASE.  (See imagezip commands below.)  
          Make sure the Operational Mode is MINIMAL in the Image Descriptor, rather than NORMALv2!

      - Make a firewalled experiment using the WINXP-BASE image above, log in as "root".

         . The experiment should be behind a firewall, to avoid contamination.

            # Firewall while making Windows images.
            set fw [new Firewall $ns]
            $fw set-type ipfw2-vlan
            $fw set-style basic

         . Note that it takes a couple of minutes after booting for the RDP service to start,
           so don't worry if you can't log in at first.

      - Set the Windows "w32time" NTP client to connect to the Emulab NTP host.
        Runs as a service, periodically contacts the time server.
          # ntp1 is a DNS alias for Ops.
          ### Do this in a Windows CMD shell, since you don't have Cygwin shells yet. ###
          net time /querysntp
          # Need to restart w32time before it sees the setsntp configuration. (?)
          net stop w32time
          net time /setsntp:ntp1
          net time /querysntp
          net start w32time
          # May take a minute to take effect.
             
      - Disable the Messenger Service to keep annoying spam pop-ups away,
        and to foil the Mailslot Heap Overflow Vulnerability, CVE-2006-1314
        Also the Alerter Service, which also uses Mailslots on TCP port 445.
        See http://www.microsoft.com/technet/security/bulletin/ms06-035.mspx
            # Optional informational commands are indented...
            cygrunsrv -VQ  Messenger
          sc config Messenger start= disabled
          sc stop Messenger
            cygrunsrv -VQ  Alerter
          sc config Alerter start= disabled
          sc stop Alerter

      - Disable the SSDP Discovery Service and Universal Plug and Play Device Host.
        This closes port 5000 to attacks.  Also disable the Remote Registry service.
            cygrunsrv -VQ  SSDPSRV
            cygrunsrv -VQ  upnphost
            cygrunsrv -VQ  RemoteRegistry

          sc config SSDPSRV start= disabled
          sc config upnphost start= disabled
          sc config RemoteRegistry start= disabled

          sc stop SSDPSRV
          sc stop upnphost
          sc stop RemoteRegistry

      - Go into Control Panel/Administrative Tools (it's under Performance and
        Maintenance in the new Control Panel interface.)

        . Right-click Start/"Explore All Users" and drag a copy of the Computer
          Management shortcut from Administrative Tools into the All Users/Desktop
          folder.

      - If you haven't already made a "root" account, go into Computer Management/
        System Tools/Local Users and Groups/Users, put it in the Administrators group.
        . Also make it a member of the Users groups.

      - Start IE, make "blank" the home page. Click Tools/Internet Options/Home page/Use Blank.

      - Show My Computer.  (Desktop Properties/Desktop/Customize Desktop...)
        Turn off "Run Desktop Cleanup Wizard every 60 days".

      - Log in as root and create C:/Temp, C:/Software/Cygwin
          ### [Do this in a Windows cmd prompt, there's no Cygwin shell yet...] ###
          mkdir C:\Temp
          mkdir C:\Software\Cygwin

      - Install Cygwin
        . (Try copying /etc/setup/* from an existing image to script the selection work.)
           # There's no scp to copy them with, so use Windows SMB File Sharing to get it
           # from \\fs\share\windows\cygwin-etc-setup.)
             mkdir C:\cygwin
           # Hm.  Still didn't come up with the added packages selected.
           # When C:\Software\Cygwin/http... is copied, it thinks everything is already installed.
           # Removing the /etc/setup/installed.db file seems not to help.
           # Nothing relevant to install in HKLM/SOFTWARE/Cygnus Solutions/Cygwin, either...

        . Download setup from www.cygwin.com/setup.exe to C:/Software/Cygwin.
          I keep a copy cached on /share/windows.  
          When you already have a Cygwin, you can update the setup.exe by:

           # [On Ops as you.]
           set pc=253
           scp -p /share/windows/cygwin-setup.exe pc$pc":"/tmp/setup.exe
             # Update the cached copy if you downloaded a new one.
             scp -p pc$pc":"/cygdrive/c/Software/Cygwin/setup.exe /share/windows/cygwin-setup.exe

           # [On the machine as root.]
           csc=C:/Software/Cygwin
           chown -R root.None $csc
           chmod g+w $csc/setup.exe
           cp -p /tmp/setup.exe $csc
           chmod 755 $csc/setup.exe
           ls -l $csc
           $csc/setup.exe &

        . Run Cygwin setup.exe in an RDP window.  (GUI-only interface.)
            # When Cygwin is already installed, ensure that upgrading SSH won't hang.
            net stop sshd
              ps -Welf | grep 'ssh'
              killall ssh-agent

            # Ditto agetty.  The SysV init daemon keeps it running.
              ps -Welf | grep 'agetty\|init'
              cygrunsrv.exe -VQ init
            net stop init
              ps -Welf | grep 'agetty\|init'
              cygrunsrv.exe -VQ init
            mv /sbin/agetty.exe{,.save}

            C:/Software/Cygwin/setup.exe &

        . Install dir is C:\cygwin, package dir is C:\Software\Cygwin .
        . Download site mirror is http://mirrors.xmission.com .
        . Click [View] to "Not Installed" (alphabetical), or "Partial" to see an update.
        . Click on the Skip in the "New" column and change it to add a binary version of:
            [agetty and more are now in the util-linux package.]
            agetty (with src), bison, cvs, cygrunsrv, ed, file, flex, gawk, gcc,
            gdb, inetutils, make, minires-devel, more, nano, netcat, openssh (with src),
            openssl-devel, patch, perl, perl-libwin32, ping, psmisc, python, rpm, rsync, rxvt, 
            shutdown, sysvinit, tcsh, vim, wget, zip .
          Click in the "src" column for openssh and agetty, so patches can be applied.
        . Since installs hang on /sbin or /etc files, open the permissions up very temporarily.
            chmod 777 /sbin /etc
          Then click "Next" to do the installations.
            # Close up again afterward.
            chmod 755 /sbin /etc
        . At the end, don't "Create an icon on the Desktop", do "Add icon to Start Menu".
        . A Cygwin update will probably tell you to reboot to get new dll's loaded.

        . Add ;C:\cygwin\bin to the end of the System PATH in 
          Control Panel/System/Advanced/Environment Variables.

        . Start up a Cygwin shell and fix the shell properties (right-click title bar):
            Options QuickEdit Mode on, Layout/screen buffer height 3000, window height 55.
            Check OK, "Modify shortcut that started this window", OK.
          - Might as well fix the Start/Programs/Accessories/Command Prompt properties, too.
          - Create c:\cygin\cygwin-tcsh.bat as a copy of c:\cygin\cygwin.bat with
            "bash --login -i" changed to "tcsh -l".
               ### Now we can use Cygwin (Bash) shell commands... ###
               alias v='ls -lsF'
               cd C:/cygwin
               sed 's/bash --login -i/tcsh -l/' < cygwin.bat > cygwin-tcsh.bat
               chmod +x cygwin-tcsh.bat
          - Copy the shortcuts in \\fs\share\windows\Desktop to the All Users/Desktop
            . Make sure the directory winds up with owner and group SYSTEM.SYSTEM,
              and the shortcut (.lnk) files Administrator.none .
                audtop=/cygdrive/c/Documents\ and\ Settings/All\ Users/Desktop
                  v -d "$audtop"
                  v "$audtop"
                chown -R root "$audtop"
                cp /share/windows/Desktop/*.lnk "$audtop"
                chmod 755 "$audtop"/*
                chown Administrator.None "$audtop"/*
                chmod 755 "$audtop"
                chown SYSTEM.SYSTEM "$audtop"
                  v -d "$audtop"
                  v "$audtop"
                # Put the Shell icons into All Users/Start Menu/Programs/Cygwin as well.
                auspc=/cygdrive/c/Documents\ and\ Settings/All\ Users/Start\ Menu/Programs/Cygwin
                  v -d "$auspc"
                  v "$auspc"
                chown root "$auspc"
                cp "$audtop"/*Shell.lnk "$auspc"
                chmod 755 "$auspc"/*
                chown Administrator.None "$auspc"/*
                chmod 755 "$auspc"
                chown Administrator.None "$auspc"
                  v -d "$auspc"
                  v "$auspc"
            or:
            . Copy the bash shortcut in Programs to the All Users/Desktop.  
            . Copy it to a tcsh icon as well, changing the Target to c:\cygin\cygwin-tcsh.bat .

        . Make rxvt All Users desktop icons for both Bash and TCSH.  
          It doesn't need an X server running, but uses the X mouse cut-and-paste keys.
          Windows shortcut options such as layout are ignored.  Use dash args or ~/.Xdefaults.
              Name: rxvt Bash  Shell
              Target: C:\cygwin\bin\rxvt.exe -g 80x55 -sl 3000 -e /bin/bash -l
              Start in: C:\cygwin\bin

              Name: rxvt TCSH Shell
              Target: C:\cygwin\bin\rxvt.exe -g 80x55 -sl 3000 -e /bin/tcsh -l
              Start in: C:\cygwin\bin


        . Set up local homedirs under /home as a symlink.  ~root is already there.
            cd /tmp
            mv /home{,.orig}
            ln -s /cygdrive/c/Documents\ and\ Settings/ /home
          Get the root dotfiles
              ls -l //fs/share/windows/cygwin.root*
              ls -l ~root/.??*
            for f in //fs/share/windows/cygwin.root*; do
              rf=`echo $f | sed s=.*cygwin.root=~root/=`
              echo $f $rf; cp -p $f $rf; chown root $rf
            done
              ls -l ~root/.??*

        . The Windows hosts file should already be symlinked into the Cygwin /etc.
            ls -l /etc/hosts
              ln -s /cygdrive/c/WINDOWS/system32/drivers/etc/hosts /etc/hosts

        . Create a proper group file.  Make wheel an alias for Administrators.
            mkgroup -l | \
              awk '/^Administrators:/{print "wheel" substr($0, index($0,":"))} \
                   {print}' > /etc/group.new
            diff /etc/group{,.new}
            cp -p /etc/group{,.prev}
            mv /etc/group{.new,}

        . Update the passwd file after creating new accounts.  Make root uid 0 with /home/root.
            mkpasswd -l | awk -F: 'BEGIN{ OFS=":" } \
               { if ($1=="root") $3="0"; \
                 else if ($1=="sshd") $NF="/bin/false"; \
                      else sub("/home/", "/users/"); \
                 print }' > /etc/passwd.new
            diff /etc/passwd{,.new}
            cp -p /etc/passwd{,.prev}
            cp -p /etc/passwd{.new,}
            chown root /etc/{passwd,group}*

            ### Note: the root UID changed from 1003 to 0 in this process.  ###
            ### Restart your Bash shell to get the new one before going on!  ###
            id

        . Set up the syslog daemon.  (See usr/share/doc/Cygwin/inetutils-1.3.2.README)
            # Make sure /etc isn't owned by SYSTEM, which will prevent making syslogd.conf .
            chown root /etc
            syslogd-config -y
            # Start the daemon.  It starts automatically at reboot.
            net start syslogd
              # Test.
              logger "Test syslogd."
              tail /var/log/messages

        . Set up sshd.  
          - Edit /bin/ssh-host-config to add a -i argument to the "cygrunsrv -I sshd" lines.
                grep cygrunsrv /bin/ssh-host-config | grep -e -I
              ed /bin/ssh-host-config
              1
              /cygrunsrv -I sshd/s//& -i/p
              /cygrunsrv -I sshd/s//& -i/p
              w
              q
          - Then stop sshd and remove its entry (if previously installed), run ssh-host-config:
            (You must be logged in as root over RDP, not ssh when you do this!)
                cygrunsrv -VQ sshd
              net stop sshd
              cygrunsrv -R sshd

              # May need to do some unmounts before running ssh-host-config.
              # (It does a mount, and there's a hard-wired limit of 31 mount table entries.)
              mount | wc -l
                ## mount: /ssh-host-config.3048: Too many mount entries
                for s in /users/s*; do umount $s; done

              # Should be NO ssh processes running, not even ssh-agent!
              ps -Welf | grep ssh

              # Make sure /etc is writable by root.
                v -d /etc
              chown root /etc

              ssh-host-config -y -c "ntsec tty"
              # or run ssh-host-config without args and answer the following interactive questions:
              # Select privilege separation = yes, sshd user = yes, install as service = yes, 
              # CYGWIN=ntsec tty

                v /etc/ssh*_config
              chown SYSTEM /etc/ssh*_config
              chmod 644 /etc/ssh*_config

          - Check for -i flag: look for Interactive = 0x00000001 (1)
              regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/sshd/Parameters
                # Also shows up under Process Type.
                cygrunsrv -VQ sshd

          - Edit /etc/sshd_config
            . Add AuthorizedKeysFile paths under /sshkeys/%u .
                  grep AuthorizedKeysFile /etc/sshd_config
                # Make it writable to edit, then change it back.
                chmod g+w /etc/sshd_config
                ed /etc/sshd_config
1
/AuthorizedKeysFile
a
AuthorizedKeysFile /sshkeys/%u/authorized_keys
AuthorizedKeysFile2 /sshkeys/%u/authorized_keys2
.
w
q
                chmod g-w /etc/sshd_config

                  # Get a running sshd to read the config file with SIGHUP.
                  kill -HUP `cat /var/run/sshd.pid`

            . ## sshd service debugging. ##
              LogLevel defaults to INFO, can be set to VERBOSE, DEBUG1, etc.
              With the syslogd service running, debug events are logged to /var/log/messages .
              [Otherwise, they show up under Event Viewer / Application / sshd,
               with one line per event (ugh.)  Refresh to see new events with F5.]
                ls -l /etc/sshd_config
                # Check.
                grep LogLevel /etc/sshd_config
                # Make it writable to edit, then change it back.
                chmod g+w /etc/sshd_config
                ed /etc/sshd_config
/#LogLevel/a
LogLevel DEBUG2
.
w
q
                chmod g-w /etc/sshd_config
                # Get a running sshd to read the config file with SIGHUP.
                kill -HUP `cat /var/run/sshd.pid`

          - Check /var/empty to avoid this error:
              /var/empty must be owned by root and not group or world-writable.
            # Actually, it must be owned by SYSTEM.
              v -d /var/empty
            chown SYSTEM /var/empty
            chmod go-w /var/empty

          - You can avoid patching and rebuilding sshd.exe if there's one saved.
              # Currently 4.3p2-3.
              cygcheck.exe -c openssh
              v //fs/share/windows/sshd.exe
              v /usr/sbin/sshd.exe

              mv /usr/sbin/sshd.exe{,.orig}
              cp -p //fs/share/windows/sshd.exe /usr/sbin/sshd.exe

          - Install the Emulab standard host keys, dated Jun 21  2001.
              ls -l /etc/ssh*
              ls -l //fs/share/windows/elab-host-keys
              ls -l /etc/ssh*key*
              ls -l /etc/orig-ssh-keys

              mkdir /etc/orig-ssh-keys
              chown root /etc/ssh*key*
              cp -p /etc/ssh*key* /etc/orig-ssh-keys
              chown SYSTEM /etc/orig-ssh-keys/*
              ls -l /etc/orig-ssh-keys

              cp -p //fs/share/windows/elab-host-keys/* /etc
              chown SYSTEM /etc/ssh*key*
              ls -l /etc/ssh*key*

          - Install the Emulab ssl certs too.
                ls -l /etc/emulab
              mkdir /etc/emulab
                ls -l //fs/share/windows/elab-ssl-certs/* /etc/emulab/*.pem
              cp -p ~/elab-ssl-certs/* /etc/emulab

          - Get an ssh pubkey to authenticate root ssh access from Boss.
            The Emulab rc.localize script over-writes this, with the same value from the DB.
                v -d /sshkeys /sshkeys/root /sshkeys/root/authorized_keys
              # Make an sshkeys directory, with a subdir for root.
              mkdir -p /sshkeys/root
              chmod 777 /sshkeys
              chmod 700 /sshkeys/root
                v /sshkeys/root
                cat /sshkeys/root/authorized_keys

              # Cut and paste the Boss root RSA pubkey (SSH version 2) to the authkeys file.
              # [On boss, copy this.]
              sudo cat ~root/.ssh/id_rsa.pub
              # [On the node.]
              cat > /sshkeys/root/authorized_keys
              # Paste, hit ^D for EOF.
              v /sshkeys/root/authorized_keys
              chmod 600 /sshkeys/root/authorized_keys

          - Start sshd.
              net start sshd
              tail /var/log/messages
          
          - Verify that we can ssh in as root from boss, with no password prompt.
              rootpc $pc id

      - Install tools: WinZip and Emacs.
            # [On boss:]
            sudo scp -rp /share/windows/emacs-21.3-fullbin-i386.tar.gz root@pc$pc":"/tmp
            sudo scp -rp /share/windows/winzip90.exe root@pc$pc":"/tmp

            # Log in as root via RDP.
            rootrd $pc
            # [On the node, as root.]
            # Graphical installer.  Start with WinZip Classic, custom setup, no desktop icon.
            /tmp/winzip90.exe

            cd C:
            # Don't worry about a plethora of "Cannot change ownership" warnings.
            tar xfoz /tmp/emacs-21.3-fullbin-i386.tar.gz
            # Graphical; click OK to set up the registry, start menu, etc.
            C:/emacs-21.3/bin/addpm.exe

            # Then copy the Emacs shortcut to the All Users/Desktop folder.
            allusers=/cygdrive/c/Documents\ and\ Settings/All\ Users
            chown root "$allusers"/Desktop
            cp "$allusers"/{Start\ Menu/Programs/Gnu\ Emacs,Desktop}/Emacs.lnk
            chown SYSTEM "$allusers"/Desktop

            # Make "emacs" be the NTEmacs runemacs starter, with "emacs-exe" for a compiler.
            ln -s /cygdrive/c/emacs-21.3/bin/runemacs.exe /usr/local/bin/emacs
            ln -s /cygdrive/c/emacs-21.3/bin/emacs.exe /usr/local/bin/emacs-exe

      - Get other stuff that "make client" depends on.

                ## Collect the include files for mysql and the Boost Graph Library on Boss.
                cd /usr/local/include
                tar cfz /share/windows/mysql-include.tgz mysql
                tar cfz /share/windows/boost-include.tgz boost
                ## Elvin distribution.
                  v /usr/testbed/www/distributions/*elvin*-4.0.3.tar.gz
                cp -p /usr/testbed/www/distributions/*elvin*-4.0.3.tar.gz /share/windows

            mkdir /usr/local/include
            cd /usr/local/include
            tar xfoz //fs/share/windows/mysql-include.tgz
            tar xfoz //fs/share/windows/boost-include.tgz

            # Build Elvin libs with GCC for testbed client programs.  
            # Need a path without embedded spaces for the make actions to work.
            mkdir C:/elvin
            cd C:/elvin
            # Don't worry about a plethora of "Cannot change ownership" warnings.
              v //fs/share/windows/*elvin*-4.0.3.tar.gz
            tar xfoz //fs/share/windows/libelvin-4.0.3.tar.gz
            tar xfoz //fs/share/windows/elvind-4.0.3.tar.gz

            cd C:/elvin/libelvin-4.0.3
              # configure: error: Elvin requires that doubles be IEEE 754 compliant
              # Edit configure, line 3547, add exit(0); to patch around it.
              ed configure
3546p
a
exit(0);
.
w
q
            ./configure >& configure.trace 
            # Ends with "creating src/include/elvin/config.h"
            tail configure.trace

            # Comment this out in c:/elvin/libelvin-4.0.3/src/lib/i18n.c :
                      #elif defined(HAVE_WINBASE_H)
                              FreeLibrary(cat);
            ed c:/elvin/libelvin-4.0.3/src/lib/i18n.c
            /HAVE_WINBASE/p
            .,.+1s|^|//|p
            w
            q

            make >& make.log1
            tail make.log1
            make install >& install.log1
            tail install.log1
              make clean

# [ SKIP once you have a WIN32 Elvin tarball in /share/windows.
            # Build Elvin for Windows on Coke, and tar it up for later installation.
            scp -p bos:"/usr/testbed/www/distributions/*elvin*-4.0.3.tar.gz" /tmp
            mkdir C:/elvin
            cd C:/elvin
            tar xfoz /tmp/libelvin-4.0.3.tar.gz
            tar xfoz /tmp/elvind-4.0.3.tar.gz
            # Rename lib dir for makefiles in elvind.
            mv libelvin-4.0.3 elvin4

            cd C:/elvin/elvin4
            nmake /k /f Makefile.win >& lib-make.winlog1
            mkdir -p C:/Program\ Files/elvin4/{bin,lib,doc}
            cp -p win32/bin/*.exe C:/Program\ Files/elvin4/bin
            cp -p win32/lib/{,*/}*.{dll,lib} C:/Program\ Files/elvin4/lib
            mkdir C:/Program\ Files/elvin4/include
            cp -p src/include/elvin/*.h C:/Program\ Files/elvin4/include

            cd C:/elvin/elvind-4.0.3        
            nmake /k /f Makefile.win >& program-make.winlog1
            cp -p *.exe *.pem C:/Program\ Files/elvin4/bin
            cp -p [A-Z][A-Z]* C:/Program\ Files/elvin4/doc
              scp -p ../*/*.winlog* ops:/proj/testbed/fish/elvin
            scp -p ops:/proj/testbed/fish/elvin-config /cygdrive/c/Program\ Files/elvin4/bin

            # Install dll's in the system so the server can be run.
            v C:/Program\ Files/elvin4/lib
            chmod -R g-w C:/Program\ Files/elvin4
            chmod a+x C:/Program\ Files/elvin4/lib/*
            cp -p C:/Program\ Files/elvin4/lib/* $nts

            elvin="C:/Program Files/elvin4/bin/elvinsvc.exe"
              v "$elvin"
            "$elvin" --help
            # Application Error - The application failed to initialize properly (0xc0000022).

            tar cfz /tmp/elvin4-windows.tar.gz -C /cygdrive/c Program\ Files/elvin4
            scp -p /tmp/elvin4-windows.tar.gz ops:/share/windows
# SKIP ]

            # Install the Windows Elvin, built on Coke above.
            # [On the experiment node as root (Bash shell):]
            rootpc $pc
              cd C:
                ls -ld Program\ Files/elvin*
              # Don't worry about a plethora of "Cannot change ownership" warnings.
              tar xvfz //fs/share/windows/elvin4-windows.tar.gz
              chown -R root Program\ Files/elvin4
              cp -p C:/Program\ Files/elvin4/lib/* C:/WINDOWS/system32
              cp -p C:/Program\ Files/elvin4/lib/* /usr/local/lib
                diff /usr/local/etc/elvind_ssl.pem C:/Program\ Files/elvin4/bin/elvind_ssl.pem
              cp -p C:/Program\ Files/elvin4/bin/elvind_ssl.pem /usr/local/etc/elvind_ssl.pem

              elvind="C:/Program Files/elvin4"
              elvin="$elvind/bin/elvinsvc.exe"
                ls -l "$elvind/bin"
              chmod -R g-w "$elvind"
                ## Graphical help message.
                "$elvin" --help &
              # Install as a service.
              "$elvin" -r
              # Install a config file and set the path for the server.
                diff /usr/local/etc/elvind.conf //fs/share/windows/elvind.conf
              cp //fs/share/windows/elvind.conf /usr/local/etc/elvind.conf
                ls -l /usr/local/etc/elvind.conf
              # Do once to register the config file.
              "$elvin" -c `cygpath -w /usr/local/etc/elvind.conf`
                  ## Testing: start elvinsvc from the Services Manager now.
                  net start elvinsvc.exe
              # Make elvinsvc automatic in services manager, or use these commands:
                regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/elvinsvc.exe
              # (4 is Disabled, 3 is Manual, 2 is Automatic, 1 is only used for System services.)
              regtool -v set /HKLM/SYSTEM/CurrentControlSet/Services/elvinsvc.exe/Start 2

# [ SKIP testing
              ## Use any Windows experiment with a Program object in it for testing.
              pid=testbed eid=Windows-1
                pid=testbed eid=Windows-1b
                pid=testbed eid=Windows-1c
              $BINDIR/evproxy -s event-server -e $pid/$eid
              
              ## program-agent debugging.
                ps -Welf | grep program-agent
                $rc/rc.progagent shutdown
              $rc/rc.progagent boot
                ## Debugging.
                tail $LOGDIR/progagent.debug
                program-agent -d -e $pid/$eid -s localhost -c /var/emulab/boot/progagents
                # [On ops.]
                tevc -e testbed/Windows-1c now prog0 start \
                    COMMAND="bash -c 'date; hostname' > /tmp/host.txt"
                # [On the node.]
                tail /tmp/host.txt
                cat /local/logs/prog0.status
              
                ## C:\cygwin\bin\tcsh.exe (2504): *** couldn't create window, Win32 error 5
                ## See http://comments.gmane.org/gmane.os.cygwin.patches/2559
                ## This is at cygwin-1.5.17-1-winsup/cygwin/window.cc:wininfo::winthread():96
                ## Try starting rc.progagent as a separate service with -i for a desktop.
                
                  ## Started up and stopped immediately.  Needs something else in rc.bootsetup.
                  --dep elvinsvc.exe \
                
                  ## Depend on EmulabStartup (rc.bootsetup), which depends on the elvin service,
                  ## and also starts evproxy.  But it stops rather than staying running...
                  --dep EmulabStartup \
                
                ## Make it manual, and explicitly start it after rc.bootsetup in EmulabStartup.
                ## Works, but stays in "starting" state, err in bootsetup.log:
                ##  cygrunsrv: Error starting a service: QueryServiceStatus:  Win32 error 1053:
                ##  The service did not respond to the start or control request in a timely fashion.
# SKIP ]

              # For setuid() to work, Root must have these rights: Create a token object; Replace a
              # process level token; and Increase Quota rights.
              # http://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-switch,
              # http://msdn.microsoft.com/library/en-us/secauthz/security/authorization_constants.asp
              editrights -u root -l
              editrights -u root -a SeCreateTokenPrivilege -l
              editrights -u root -a SeAssignPrimaryTokenPrivilege -l
              editrights -u root -a SeIncreaseQuotaPrivilege -l

              # Set up to run the program-agent service.  Notice the -i (interactive) argument.
                cygrunsrv -R ProgAgent
              progagent=/usr/local/etc/emulab/rc/rc.progagent
              proglog=/var/log/program-agent.log
              cygrunsrv -I ProgAgent -d "Emulab Program Agent" -i \
                  --type manual \
                  -p /cygdrive/c/cygwin/bin/bash \
                  -a "--norc --noprofile -c '$progagent >& $proglog &'"
                regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/ProgAgent/Parameters
                cygrunsrv -VQ ProgAgent
                  ## This won't work until you build the Emulab programs, including program-agent, below.
                  cygrunsrv -S ProgAgent
                  cygrunsrv -E ProgAgent
                # Log files.
                  tail /var/log/{program-agent,ProgAgent}.log
                touch /var/log/{program-agent,ProgAgent}.log
                chmod 777 /var/log/{program-agent,ProgAgent}.log

                # Messages go to syslog.
                tail /var/log/messages

# [ SKIP testing
                # Little problem: "Must be root to run this script!"
                # Add this: 
                  # This runs as a separate Local System service on XP.  Change to root.
                  if (WINDOWS()) { $EUID = $UID = 0; }

                # Testing on ops.
                tevc -e testbed/bsd-1 now prog0 start
                tevc -e testbed/bsd-1 now prog0 start COMMAND='hostname >>& /users/fish/test.out'

                tevc -e testbed/Windows-1b now prog0 start COMMAND='hostname>>&/users/fish/test.out'
                  v /users/fish/test.out
                  tail /users/fish/test.out
                tevc -e testbed/Windows-1b now prog0 run COMMAND='touch /tmp/foo'
                tevc -e testbed/Windows-1b now prog0 run COMMAND='id'
                tevc -e testbed/Windows-1b now prog0 run COMMAND='ls -l /users/fish'
                tevc -e testbed/Windows-1b now prog0 run COMMAND='ls -l /proj/testbed/fish'

                # [On the node.]
                cat /local/logs/prog0.status
                cat /local/logs/prog0.err
                cat /local/logs/prog0.out
# SKIP ]

      - Get the testbed client code via CVS and rsync, build, and install it.

            #[On boss.]
            set pc=254
              sudo ssh pc$pc id
            sudo ssh pc$pc ls -l flux
              sudo ssh pc$pc mkdir flux

            # Check out a testbed source tree somewhere on boss.
            cd /home/fish/testbed-current
              # First time only
              mkdir CVS; touch CVS/Entries; echo . > CVS/Repository
                # Simulated top setup if there's already a tree underneath.
                echo cvs.flux.utah.edu:/usr/flux/CVS > CVS/Root
                echo D/testbed//// > CVS/Entries
            # Any time the testbed tree needs to be re-created.  (Takes a while.)
            cvs -Q co testbed
              # Updates After that.
              cat CVS/Entries
              cat testbed/CVS/Entries
                cvs -n -q update testbed
              cvs -q update -d testbed

            # rsync over to the node.  Be patient, it's 50 megs.
            sudo rsync -rl testbed root@pc$pc":"flux

            # Push the downloaded binary programs into the source tree for install.
            sudo scp -p /share/windows/{WSName,addusers,usrtogrp,setx,devcon}.exe \
                        pc$pc":"flux/testbed/tmcd/cygwinxp

            #[On the node.]

              ## Initial setup stuff.

              # Install some dotfiles for Root.
              cp -p testbed/tmcd/cygwinxp/cygwin.root.bashrc ~root/.bashrc
              cp -p testbed/tmcd/cygwinxp/cygwin.root.bash_profile ~root/.bash_profile
              cp -p testbed/tmcd/cygwinxp/cygwin.root.emacs ~root/.emacs
              # No HOME envar is set for root's desktop, so Emacs defaults it to C:/ .
              cp -p ~root/.emacs C:/.emacs

              # Install site-lisp files for Emacs.
                v -t testbed/tmcd/cygwinxp/site-lisp
                v -t c:/emacs-21.3/site-lisp
              cp -rp testbed/tmcd/cygwinxp/site-lisp/* c:/emacs-21.3/site-lisp
              ls -l c:/emacs-21.3/site-lisp

              # Need a resolv.conf before tmcc will work.
                cat /etc/resolv.conf
              cp -p ~/flux/testbed/tmcd/cygwinxp/resolv.conf /etc/resolv.conf

              # The Elvin and program-agent man pages get installed in man8.
              mkdir /usr/local/man/man8

            # Finally ready to do the Emulab makes!
            mkdir ~/flux/obj-real
            cd ~/flux/obj-real

            # Configure takes a while...
              v configure.trace*
              mv configure.trace{,.1}
            ../testbed/configure --enable-windows --enable-windowsclient >& configure.trace
            # Should end with "creating config.h".
            tail configure.trace
              tail -f configure.trace

            # If this is an update, evproxy is run by rc.bootsetup and nothing stops it.
            # The install of evproxy in the make will fail unless we stop it first.
            ps -Welf | grep evproxy
            killall evproxy
              kill `ps -Welf | grep evproxy | awk '{print $2}'`
            # Ditto emulab-syncd, program-agent, and slothd.
              ps -Welf | grep emulab
            $rc/rc.syncserver shutdown
            $rc/rc.progagent shutdown
            $rc/rc.slothd stop
            $rc/rc.linktest stop
            killall idlemon
              ps -Welf | grep emulab
              kill `ps -Welf | grep idlemon | awk '{print $2}'`

              ls -l ~/flux/testbed/tmcd/cygwinxp/*.exe
            make client-install >& make.log1
            tail make.log1

            # Install the iperf HTML doc, since the makefiles don't do it.
            idoc=/usr/share/doc/iperf
              v $idoc
            mkdir $idoc; chmod 755 $idoc
            (cd ~root/flux/testbed/event/linktest/iperf/iperf-2.0.2/doc; \
             for f in [a-z]*; do /usr/bin/install -c -o root -m 0644 $f $idoc ; done)
            v $idoc

                # Only needed if there are problems...
                  v -t make.log*
                make client-install >& make.log2
                make client-install >& make.log3
                make client-install >& make.log4
                make client-install >& make.log5
                make client-install >& make.log6
                make client-install >& make.log7
                make client-install >& make.log8
                make client-install >& make.log9

              # Optional: save the Emulab client-side make tree.  'prepare' wipes it out.
              #[On the node as me.]
              cd ~root
                v $ptf/xpimage/*.tgz
                mv $ptf/xpimage/xpimage-elab{,.3}.tgz
              tar cfz $ptf/xpimage/xpimage-elab.tgz flux
                #[Best done on Ops.]
                tar tvfz $ptf/xpimage/xpimage-elab.tgz | tail

              # Re-install later.  Copy in on the node as me.
                v ~root/flux
                v $ptf/xpimage/xpimage*.tgz
              cp $ptf/xpimage/xpimage-elab.tgz /tmp

              #[On the node as root.]
              cd ~root
                v flux
                tar tvfz /tmp/xpimage-elab.tgz | head
              tar xfoz /tmp/xpimage-elab.tgz flux

      . Patch the /etc/profile file to use /home dirs if the /users mounts are down.
        ## Note their comment:
        # IF THIS FILE IS MODIFIED IT WILL NOT BE UPDATED BY THE CYGWIN
        # SETUP PROGRAM.  IT BECOMES YOUR RESPONSIBILITY.
        #
        # The latest version as installed by the Cygwin Setup program can
        # always be found at /etc/defaults/etc/profile
            # If you need to check in a new version back in CVS land...
            (cd ~/flux; cvs update testbed/tmcd/cygwinxp/profile)
          diff /etc/defaults/etc/profile ~/flux/testbed/tmcd/cygwinxp
            # If the diffs are right, just copy the Emulab one.
            cp ~/flux/testbed/tmcd/cygwinxp/profile /etc
          # Otherwise, edit the file.
            diff /etc/defaults/etc/profile /etc/profile
            cp /etc/defaults/etc/profile /etc
          ed /etc/profile
  /^# If the home directory doesn't exist, create it./,/^if \[ ! -d "\${HOME}" \]; then/p
/^# If the home directory doesn't exist, create it./,/^if \[ ! -d "\${HOME}" \]; then/c
### Use a local dir under sshd if the mount failed.
if [ ! -d "$HOME" ]; then
        HOME=/home/$USER
fi
# If the home directory doesn't exist, create it.
if [ ]; then
###if [ ! -d "${HOME}" ]; then
.
  .-10,.+5p
  w
  q

      . Need an NTP client, or at least the semblence of one.
            cat /etc/ntp.drift
          echo 0.000 > /etc/ntp.drift

      . Set up the tbshutdown script to run as a service, to get a shutdown signal.
            editrights -u root -l
          editrights -u root -a SeServiceLogonRight -l
          # Don't forget to set the root password to the following, if you haven't done it yet.
          rootpwd='myRootPwd'
          # EmulabShutdown is started manually later on from rc.cygwinxp .
          echo "$rootpwd"
            cygrunsrv -R EmulabShutdown
          cygrunsrv -I EmulabShutdown -u root -w "$rootpwd" -p /cygdrive/c/cygwin/bin/bash \
              --shutdown --type manual \
              -a "--norc --noprofile -c '/usr/local/etc/emulab/tbshutdown'"
          cygrunsrv -VQ EmulabShutdown

          # If you see the following, try running rc.accounts or rc.bootsetup below to 
          # clear it up.  Haven't figured this out yet...
          ##cygrunsrv: Error installing a service: CreateService:  Win32 error 1057:
          ##The account name is invalid or does not exist, or the password is invalid 
          ##for the account name specified.

          # Log files
          touch /var/log/EmulabShutdown.log
          chmod 666 /var/log/EmulabShutdown.log
          regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/EmulabShutdown/Parameters
            # Manual start-up for testing.
            cygrunsrv -S EmulabShutdown

      . See if rc.bootsetup works.
          # Don't allow the sshd uid a shell login in rc.accounts/liblocsetup.pm::os_accounts*().
          ## Not needed, user sshd isn't allowed to login anyway.  "Permission denied."
            grep '\^sshd:' /var/emulab/boot/usershells
          echo '/^sshd:/s|/bin/bash$|/bin/false|' >> /var/emulab/boot/usershells

          ##Running os dependent initialization script rc.cygwin
          ##chmod: cannot access `/var/log/EmulabStartup.log': No such file or directory
          ##chmod: cannot access `/etc/emulab/iscygwin': No such file or directory
          touch /var/log/EmulabStartup.log
          chmod 666 /var/log/EmulabStartup.log

          # May need to make /sshkeys, if it hasn't been done above.
          v -d /sshkeys
            mkdir /sshkeys
            chmod 777 /sshkeys

              ## May need to read .bashrc installed above if you haven't set up the path yet.
              . ~/.bashrc
            tmcc nodeid
              ## If you have problems, you may be missing /etc/resolv.conf .
              tmcc -d nodeid
                    nodeid 
                    /usr/local/etc/emulab/tmcc.bin  -d nodeid 
                    Connection to TMCD refused. Waiting ...
            ## You can test rc.cygwin separately.  It should reboot, the first time, 
            ## when it changes the node ID.
            $rc/rc.cygwin

            ## You can try the boot-time script to see that all is well.
            ## If you run this, and the node name hasn't been changed yet, it will reboot.
            $rc/rc.bootsetup

      . Set up the boot script to run as a service.

          # Start up after DHCP, Elvin, and W32Time (for watchdog), run ProgAgent afterwards.
              cygrunsrv -R EmulabStartup 
          rootpwd='myRootPwd'
          firstboot=/usr/local/etc/emulab/rc/rc.firstboot
          bootsetup=/usr/local/etc/emulab/rc/rc.bootsetup
          sshrun="cygrunsrv -S sshd"
          progrun="cygrunsrv -S ProgAgent"
          bootlog=/var/log/bootsetup.log
          bashargs="--norc --noprofile -c"
          cygrunsrv -I EmulabStartup -u root -w $rootpwd \
            --dep DHCP --dep elvinsvc.exe --dep W32Time \
            -p /cygdrive/c/cygwin/bin/bash \
            -a "$bashargs '( $firstboot; $bootsetup; $sshrun; $progrun ) >& $bootlog'"
            ## Alternate to accumulate bootlog, rather than only saving the last one.
            -a "$bashargs '( $firstboot; $bootsetup; $sshrun; $progrun ) >> $bootlog 2>&1'"
          cygrunsrv -VQ EmulabStartup | fold -w 78

            ## If you run this, and the node name hasn't been changed yet, it will reboot.
            cygrunsrv -S EmulabStartup 
            cygrunsrv --help
          regtool -v list /HKLM/SYSTEM/CurrentControlSet/Services/EmulabStartup/Parameters
          sc query EmulabStartup

      . Make a $HOME envar for everybody, so Emacs works on startup from the desktop.
        - Set a user environment variable: HOME = /users/%USERNAME%
        - Stored in HKCU/Environment, which is HKU/*/Environment based on the user SIDs.
        - The user registry key (folder) is created at first login, doesn't exist before that.
          Run setx after that at login time to set the HOME environment variable value.
                # Check.
                regtool get /HKLM/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/SetHOME
          # Use a literal Windows command rather than a script.
          regtool -s set /HKLM/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/SetHOME \
            'cmd /C "if not %USERNAME% == root if not %USERNAME% == Administrator setx HOME //fs/%USERNAME%"'
                # Undo.
                regtool unset /HKLM/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/SetHOME
          # Check that setx.exe is in system32.
          v C:/WINDOWS/system32/setx.exe

# [ SKIP if patched sshd.exe and agetty.exe installed above.
      . Patch sshd so that shares (including /users homedirs) work with public-key logins.
        Also touches a file when client input is received, so slothd will know.

        - RDP into a node as root and shut down all ssh processes before update.
          (Otherwise, installation of an openssh update can hang mysteriously.)
          Use Task Manager if killall and kill don't work.
            net stop sshd
            ps -Welf | grep ssh
            killall ssh-agent

        - Go through Cygwin setup and make sure everything is updated.  
              cygcheck -c openssh
              /cygdrive/c/software/cygwin/setup.exe &
              cygcheck -c openssh
          . View "Partial" will show what it wants to download and install.
          . Also select src for openssh, which goes under /usr/src .
          . When base dll's are updated, it will tell you to reboot.  Do it.

        - Install the source patches.  (Go get CVSROOT and agent keys set above first.)
            (cd ~/flux; cvs update testbed/tmcd/cygwinxp)
              (cd ~/flux; cvs co testbed/tmcd/cygwinxp)

                ## Another option is to scp the patchs in from Boss.
                mkdir -p ~/flux/testbed/tmcd
                sudo scp -rp $bftcx root@pc$pc":"flux/testbed/tmcd

          ###cd /usr/src/openssh-4.1p1-2
          ###cd /usr/src/openssh-4.2p1-1
          cd /usr/src/openssh-4.3p2-4
            v -t *.[ch] | head -30

          # Enable no-password ssh logins which can access shared homedirs.
          cp -p uidswap.c{,.orig}
            patch -p1 --dry-run < ~/flux/testbed/tmcd/cygwinxp/uidswap.c.patch
          patch -p1 -b < ~/flux/testbed/tmcd/cygwinxp/uidswap.c.patch
            diff -c uidswap.c{.orig,}

          # Enable slothd to know of the last SSH client input time.
          for f in channels.{h,c} serverloop.c; do cp -p $f{,.orig}; done
            v *.orig
            patch -p1 --dry-run < ~/flux/testbed/tmcd/cygwinxp/sshd-client-input-time.patch
          patch -p1 -b < ~/flux/testbed/tmcd/cygwinxp/sshd-client-input-time.patch
            diff -c channels.h{.orig,}
            diff -c channels.c{.orig,}
            diff -c serverloop.c{.orig,}

        - Configure.  Takes a while.
            # These are the options that contrib/cygwin/README specifies:
            prefix=/usr sbindir=/usr/sbin datadir=$prefix/share
            ./configure > configure.trace 2>&1 \
                --prefix=/usr \
                --sysconfdir=/etc \
                --libexecdir=${sbindir} \
                --localstatedir=/var \
                --datadir=${prefix}/share \
                --mandir=${datadir}/man \
                --infodir=${datadir}/info
              tail configure.trace
                tail -f configure.trace

        - Just make and install sshd.exe, assuming everything else is up-to-date.
            make sshd.exe > make.log.1 2>&1
              tail make.log.1
                tail -f make.log.1

            # Make sure sshd is closed down while installing.
            ps -Welf | grep sshd
            net stop sshd
            mv /usr/sbin/sshd.exe{,.prev}
            /usr/bin/install -c -m 0755 -s sshd /usr/sbin/sshd.exe
            net start sshd

      . Patch agetty to support 115.2kbps serial line speed.
        agetty is now in the util-linux package (replaced by the Linux/Sun version.)
        Unfortunately, it breaks tty modes like onlcr, as well as the stty command.
        AND, it doesn't allow root logins, which makes console access kind of pointless.
        So we still need to use the old version, patched to work with 115200 bps serial.

          # [On boss:]
          sudo scp -rp /share/windows/agetty-2.1-1 root@pc$pc":"/usr/src
        #[As root, on the node.]
        cd /usr/src/agetty-2.1-1

        # Test.  If it exits after 10 seconds, turn off EMS below and reboot.
        time ./agetty -L -T vt100 -n ttyS0 115200 ttyS0
            # New getty version from Linux/Sun in the util-linux package.
            time /sbin/agetty -L 115200 ttyS0 vt100

          # Done't need to build if you can just use the executable in the directory...
          cp -p agetty.c{.orig}
            patch -p1 --dry-run < ~/flux/testbed/tmcd/cygwinxp/agetty.c.patch
          patch -p1 -b < ~/flux/testbed/tmcd/cygwinxp/agetty.c.patch
          make

          make install
        ls -l {,/sbin/}agetty.exe*
        mv /sbin/agetty.exe{,.prev}
        # Note the agetty.exe  -- The makefile doesn't get it right.
        /usr/bin/install -s -m 755 -o administrator -g administrators agetty.exe /sbin
# SKIP ]

      . Set up agetty for console logins.

        - Make sure Console Redirection is disabled after boot in the BIOS settings.

        - Turn off Windows EMS, because it disables the serial console for agetty.
          Windows "Emergency Management Services" is a second level of redirection.
          If you see the "Starting Windows" progress bar on the serial port, it's on.
          The result is that the serial port won't even show up in Device Manager.

          # Look for the EMS "OS Load Options: /redirect" and turn it off.
            bootcfg /query
            # Assuming the Boot Entry you want is #1:
            bootcfg /ems OFF /id 1
            bootcfg /query

        - Install the patched agetty, if not done above.
            #[As the user:]
            cp //fs/share/windows/agetty.exe /tmp
            #[As root:]
              mv /sbin/agetty.exe{,.prev}
            /usr/bin/install -s -m 755 -o administrator -g administrators /tmp/agetty.exe /sbin

        - Set up the sysvinit package to run agetty.

         . Uncomment the serial console line in /etc/inittab.
           Set Com1 to 115.2kbps, which is determined by the serial concentrator.
             # /etc/inittab is mode 600 SYSTEM.SYSTEM -- init is very picky about this.
             ls -l /etc/inittab
             chmod 777 /etc/inittab
               cat /etc/inittab
             ed /etc/inittab
             /agetty/s/^#//p
             /agetty/s/9600/115200/p
             w
             q
             chmod 600 /etc/inittab

         . The postinstall script does init with autoanswer=no.
               v /etc/inittab
             chown system.system /etc/inittab
             chown system.system /etc/rc
               cygrunsrv -VQ init
               cygrunsrv -R init
             # Init starts agetty from inittab.  Send it an interrupt signal at shutdown.
             cygrunsrv -I init -d "Cygwin SysV init" -p /sbin/init -a -i -s INT
             net start init

      . Make a load average log for slothd, averaged over a 1 minute period.

        - /proc/loadavg is hard-wired to "0.00 0.00 0.00" on Cygwin now.

        - All attempts to script this setup to reproduce it on another computer have
          failed so far, including using its own "Save/Restore Settings" and
          transplanting the registry subtree.

       - Click into Computer Management / Performance Logs and Alerts / Counter Logs.
          Right-click "New Log Settings..." in the logs pane, 
            Name: "ldavg", OK.

          General tab, 
            Counters list, "Add Counters...",
              check "Use local computer counters",
              click "Add" to add % total processor (the default),
              click "Close".
            Sample data every: Interval: "60" seconds.

          Log Files tab, 
            Log file type: "Text File (Comma delimited)", 
            Uncheck "End file names with" so the result goes into ldavg.csv .
            Configure... 
              Location: "C:\cygwin\var\run",
              Log file size: "Limit of: 1 MB", OK.

          Schedule tab, 
            Start Log: Click "At" (which defaults to the current time, as well as the future.) 
            Stop Log: Click "When the 1-MB log file is full.",
              When a log file closes: "Start a new log file".

          Check all three tabs, click OK.

          # ldavg should start out red (stopped) and then turn green (started) if you
          # refresh with F5.  It will start again after reboot.
          # You can turn it off and on with the right-click menu on "ldavg" in the logs pane.  
            tail -f /var/run/ldavg.csv
          # The first one is always 99.999, etc.

      . Disable TCP/IP address autoconfiguration, so unswitched interfaces like the sixth NICs
        on the pc3000's don't get Microsoft class B network 169.254 addresses assigned.
          svcs=/HKLM/SYSTEM/CurrentControlSet/Services tcpp=$svcs/Tcpip/Parameters
            regtool get $tcpp/IPAutoconfigurationEnabled
          regtool set -i $tcpp/IPAutoconfigurationEnabled 0

      . Reboot to make sure it all works right.  Note that prepare clears out the source
        trees, so don't do it until you've booted once and know that everything else works.
          prepare
          /sbin/reboot

================================================================
Making single images

    . Windows Update
      - This might be needed after each Microsoft "Patch Tuesday" (second tues of the month.)
      - After it's installed, just click Start/All Programs/Windows Update, Express, Install.
        . For initial installation, start up Internet Explorer and go to:
          http://update.microsoft.com
        . After that, Windows Update is on your Start menu.
           - The first time, just installs/updates the updater and asks to reboot.
           - I just choose the EXPRESS update, installing all high-priority updates.
           - Don't turn on Automatic Updates.

           - After rebooting, check again if there's more to install.

             To install SP2, you must have access to a console screen, because the
             Windows Firewall defaults to block both SSH and RDP.  Disable it.
             . This can be done with serial console access after the reboot with:
                   netsh firewall show config
                 netsh firewall set opmode DISABLE

             . You may need to free disk space to install SP2, or allocate a partition.
                 # The prepare script does this cleanup now.
                   v /tmp/WindowsXP-KB835935-SP2-ENU.exe
                 rm /tmp/WindowsXP-KB835935-SP2-ENU.exe
                   du -sm C:/WINDOWS/ServicePackFiles/i386
                 rm -rf C:/WINDOWS/ServicePackFiles/i386
                   du -sm C:/WINDOWS/SoftwareDistribution/Download
                 rm -rf C:/WINDOWS/SoftwareDistribution/Download/*

      - Could be a good time to update Cygwin as well.  
        . Beware of stepping on the sshd.exe and agetty.exe patches.

     . Build driver database if using sysprep.  (Only needs to be done once.)
       Set DriversPath and right-click/Install the drivers/nic .inf files
       in Windows Explorer to put them into the Drivers DB.
          regtool get /HKLM/SOFTWARE/Microsoft/Windows/CurrentVersion/DevicePath
        regtool set -s /HKLM/SOFTWARE/Microsoft/Windows/CurrentVersion/DevicePath \
                           'C:\WINDOWS\inf;C:\drivers\disk;C:\drivers\nic'
          v C:/drivers
        set sysprep=/share/windows/sysprep
        sudo scp -rp $sysprep/drivers pc$pc":"/cygdrive/c
          # You can select them all in /cygdrive/c/drivers/nic and right-click/Install once.
          # It makes corresponding PNF files in the directory and remembers DevicePath.
          e1000325.inf
          e100a325.inf
          e100b325.inf
          netag311.INF
        tail C:/WINDOWS/setupapi.log

     . Run prepare to clear out experiment-specific state.
        # Have to be on serial console to do sysprep (-s option.)
        rootpc $pc
          # Log off any non-root RDP sessions to unlock the C:/Documents and Settings files.
          qwinsta
          # Ignore complaints about all of the C:/Documents and Settings directories
          # that were never created because the users didn't log in...
          prepare -n -s

          exit

    . Add an entry at the beginning of xpimage-log.txt, and create the image
      descriptor if it's not an existing image.

    . Capture the image with imagezip.  
      You can specify the PC from which to grab the image when you create an image-id.
      Do it in red-dot mode so you can set the Reboot Waittime to 240 seconds.
      When updating existing images, I do it by hand in two stages, as below.
    
        # [On boss.]
        set pc=98 img=SP1 image=SP1-pc850_2005-09-30
        set pc=67 img=SP0 image=SP0-sysprep_2006-04-04
        set pc=167 img=UPDATE image=UPDATE-pc850_2006-02-13

        set pc=210 img=BASE-pc3000 image=BASE-pc3000_2005-08-31
        set pc=201 img=SP0-pc3000 image=SP0-pc3000_2005-09-01
        set pc=201 img=SP1-pc3000 image=SP1-pc3000_2005-09-01
        set pc=242 img=SP2-pc3000 image=SP2-pc3000_2005-09-01
        set pc=246 img=UPDATE-pc3000 image=UPDATE-pc3000_2006-02-10

        set pc=221 img=UPDATE image=UPDATE-pc3000-sysprep_2006-07-28
        set pc=106 img=UPDATE image=UPDATE-pc850-sysprep_2006-03-03

        set pc=247 img=UPDATE image=UPDATE-sysprep_2007-07-20

        df -m /proj/testbed/images /usr/testbed/images
          # Verify SSH working.
          rootpc $pc id

        # Boot into the MFS.  The serial console will show you when it's open for business.
        echo $pc $image
        node_admin on pc$pc &
          # Variant: wait until it's in the MFS before returning.
          node_admin -w on pc$pc
              # Should not be necessary if ssh from root@boss to the node is working.
              rootpc $pc /sbin/reboot
          ## You may see:
              shutdown: Couldn't reboot: Error 53 The network path was not found.
          ## Try this command:
            rootpc $pc 'tsshutdn 1 /REBOOT /DELAY:1'
          ## Or you can shutdown/restart it from the task manager under RDP,
          ## or just wait a while and Emulab will cycle power to force a reboot.

        # Wait until the node is in the MFS, based on watching the serial console.
        # Or use the node_admin -w flag to wait.
        # Then make WINXP-TMP on /proj, and move it to /usr/testbed/images with the right name.
        rootpc $pc
            cd /proj/testbed/images
            df -m /proj/testbed/images
              ls -lt /proj/testbed/images/WINXP*
            ls -l /proj/testbed/images/WINXP-TMP.ndz
          # IDE disk (pc600 & pc850, pc1 - pc178)
          imagezip -o -I 2 -I 3 -I 4 /dev/ad0 /proj/testbed/images/WINXP-TMP.ndz
          # SCSI disk. (pc3000, pc201 - pc360)
          imagezip -o -I 2 -I 3 -I 4 /dev/da0 /proj/testbed/images/WINXP-TMP.ndz
            ls -lt /proj/testbed/images/WINXP*
          exit

        # Reboot the source node back into Windows.
        echo $pc $image
        node_admin off pc$pc &

        # Move the image to /usr/testbed/images to avoid NFS reads, for faster swap-in.
          ls -l /{proj,usr}/testbed/images/WIN*
        ls -l /proj/testbed/images/WINXP-TMP.ndz /usr/testbed/images/WINXP-$image.ndz
        df -m /usr/testbed/images
        cp -p /{proj,usr}/testbed/images/WINXP-TMP.ndz
        # Check.
        ls -l /{proj,usr}/testbed/images/WINXP-TMP.ndz
        cksum /usr/testbed/images/WINXP-TMP.ndz & ssh ops cksum /proj/testbed/images/WINXP-TMP.ndz

        # Install with mv.  Frisbee might have the old inode still open.
          ls -l /usr/testbed/images/WINXP-{TMP,$image}.ndz
        mv /usr/testbed/images/WINXP-{TMP,$image}.ndz
        ls -l /usr/testbed/images/WINXP-$image.ndz
        df -m /usr/testbed/images
        # Clear the temp from /proj.
        rm -f /proj/testbed/images/WINXP-TMP.ndz
        df -m /proj/testbed/images
          ls -l /{proj,usr}/testbed/images/WIN*
        
        # Make symlinks without the date suffix, corresponding to the Image ID's.
          ## -new images for testing, before we commit.
          ls -l /usr/testbed/images/WINXP-$img-new*
          rm /usr/testbed/images/WINXP-$img-new.ndz
          ln -s WINXP-$image.ndz /usr/testbed/images/WINXP-$img-new.ndz
          ls -l /usr/testbed/images/WINXP-$img-new*
        # Production images.
          ls -l /usr/testbed/images/WINXP-$img*
        set files = `ls -F /usr/testbed/images/WINXP-$img* | grep @ | tr -d @`
        echo $files
        v $files
        foreach f ( $files )
            echo $f
            rm $f
            ln -s WINXP-$image.ndz $f
        end
          v $files
          ls -l /usr/testbed/images/WINXP-$img*

================================================================

      - Installing service packs
        Service packs are cumulative, so there's no point in installing both SP1 and SP2.

        . SP1 
          # [On boss:]
          sudo scp -rp /share/windows/xpsp1a_en_x86.exe root@pc$pc":"/tmp
          # [Log in as root via RDP.]
          rootrd $pc
            /tmp/xpsp1a_en_x86.exe &
            # Options: Do Not Archive Files
            # Reboot to finish.

        . SP2 - http://www.microsoft.com/technet/prodtechnol/winxppro/maintain/winxpsp2.mspx
          set pc=114
          sudo scp -rp /share/windows/WindowsXP-KB835935-SP2-ENU.exe root@pc$pc":"/tmp
          # [Log in as root via RDP.]
          rootrd $pc
                  # /help is SLOW.  It unpacks the component files each time you run it...
                  /tmp/WindowsXP-KB835935-SP2-ENU.exe /help &
                  # Switches are /quiet (no interaction), /passive (progress bar),
                  # /norestart (Don't boot), /n (no backup.)

              # This takes a LONG TIME to install.
              /tmp/WindowsXP-KB835935-SP2-ENU.exe /n /norestart  &

          - Turn off the Windows Firewall by default.
              # The netsh firewall commands won't work until after the reboot, and
              # then you're locked out from ssh, RDP, and even ping.  Gotta have
              # either serial or graphics console access after the reboot to turn off
              # the firewall or else you're just locked out.
                netsh firewall show config
              netsh firewall set opmode DISABLE

        . Disk space cleanup.
              v /tmp/WindowsXP-KB835935-SP2-ENU.exe
            rm /tmp/WindowsXP-KB835935-SP2-ENU.exe
              du -sm C:/WINDOWS/ServicePackFiles/i386
            rm -rf C:/WINDOWS/ServicePackFiles/i386
              du -sm C:/WINDOWS/SoftwareDistribution/Download
            rm -rf C:/WINDOWS/SoftwareDistribution/Download/*

      - Explorer/Help/About Windows (actually the 'winver' command) says this:
        . SP2 + Windows Update -
               Version 5.1 (Build 2600.xpsp_sp2_gdr.050301-1519: Service Pack 2)
        . SP2 download, on top of SP1 with no Windows Update -
               Version 5.1 (Build 2600.xpsp_sp2_rtm.040803-2158: Service Pack 2)
        . SP1, partially updated to SP2 - 
               Version 5.1 (Build 2600.xpsp2.050301-1526: Service Pack 1)
        . SP1 -
               Version 5.1 (Build 2600.xpsp1.020828-1920: Service Pack 1)
        . SP0 (no SP's) -
               Version 5.1 (Build 2600.xpclient.010817-1148)

    . DEMOTING an image to /proj/testbed/images (edit the image descriptor unless it's symlinked.)
        ls -l /usr/testbed/images/WINXP-$image.ndz
        df -m /proj/testbed/images
        cp -p /usr/testbed/images/WINXP-$image.ndz /proj/testbed/images
        # Check.
        ls -l /{usr,proj}/testbed/images/WINXP-$image.ndz
        ssh ops cksum /proj/testbed/images/WINXP-$image.ndz & cksum /usr/testbed/images/WINXP-$image.ndz
        # Clear the old copy.
        df -m /usr/testbed/images
        rm -f /usr/testbed/images/WINXP-$image.ndz
        df -m /usr/testbed/images

================================================================
Updating a set of existing images.

    . Build changes one place, make tarballs to update the others.
        #[On the build node, as me, in a Bash shell.]
          bash -l
        date=2006-11-16
        updates=/proj/testbed/fish/winxp
          v $updates/*$date*

        # First the system tarball.
        # Work around a running sshd (on the destination machines.)
        #[As root.]
        cp -p /usr/sbin/sshd.exe{,.new}
        #[Back to me.]
        cd /
        sysfiles="usr/sbin/sshd.exe.new bin/ssh-host-config etc/sshd_config \
                  etc/profile sbin/agetty.exe etc/inittab sbin/reboot \
                  usr/local/bin/install-tarfile usr/local/bin/install-rpm \
                  usr/local/bin/emulab-iperf.exe usr/share/doc/iperf"
        v $sysfiles
        # Does inittab really need to have mode 600?  Owned by SYSTEM.SYSTEM, can't read here.
        chmod 644 /etc/inittab
        tar cvfz $updates/sys_$date.tgz $sysfiles
        chmod 600 /etc/inittab

        # Then the Emulab tarball.
        cd /usr/local
        elabpat="etc/emulab man/man{1/{install,emulab}-*,8/program-*}"
        elabfiles=`eval "echo $elabpat"`
        v -d $elabfiles
          v -Rt $elabfiles | grep '~'
          v -t $elabfiles | head -20
          v -t $rc | head -20
          v /etc/emulab/paths.*
          v /usr/local/etc/emulab/{*.exe.prev,*~}
          v $rc/*~
        rm /usr/local/etc/emulab/{*.exe.prev*,*~} $rc/*~
        tar cvfz $updates/elab_$date.tgz $elabfiles

    . Install the tarballs to update a set of nodes.
        #[On Boss.]
        set date=2006-11-16
        v $rdf$date*
          script $rdf$date-sysprep
          script $rdf$date-pc850
          script $rdf$date-pc3000
          # Do the date again inside the script shell...
        set updates=/proj/testbed/fish/winxp
        set nodes="SP0 SP1 SP2 UPDATE"
          ## set nodes="node"

        # Choose ONE of the following hardware types:
          # sysprep, hardware-independent but made on the pc850.
          set hw=pc850 img=sysprep disk=/dev/ad0

          # pc850, IDE disk (pc41 - pc178)
          set hw=pc850 img=pc850 disk=/dev/ad0

          # pc3000, SCSI disk. (pc201 - pc360)
          set hw=pc3000 disk=/dev/da0

        set pid=testbed eid=new-windows-$hw
          set pid=testbed eid=new-windows-$hw"b"
          ## set pid=testbed eid=Windows-1a
          ## set pid=testbed eid=Windows-1b
          ## set pid=testbed eid=Windows-1e
          ## set pid=testbed eid=test-windows-$hw
          ## set pid=testbed eid=test-win-$hw-nofw
          ## set pid=testbed eid=test-windows-pc600
          ## set pid=testbed eid=test-windows-pc850b
          ## set pid=testbed eid=test-windows-pc3000
        set exp=$eid.$pid.emulab.net

        alias nping 'foreach node ( $nodes )\
                echo ================  $node.$exp\
                /sbin/ping -c 1 $node.$exp\
                echo ""\
              end'
        # CSH is *so* bad at this...  Have to pull off the args first.  Quoting is nasty.
        alias nodes 'set nodescmd='"'"'\!*'"'"';  foreach node ( $nodes )\
                echo ================  $node.$exp\
                ssh -n $node.$exp bash -c $nodescmd:q \
                echo ""\
              end'
        alias su_nodes 'set nodescmd='"'"'\!*'"'"';  foreach node ( $nodes )\
                echo ================  $node.$exp\
                sudo ssh -n $node.$exp bash -c $nodescmd:q \
                echo ""\
              end'

        nping
        # May update your known_hosts file the first time...
        nodes id
          su_nodes id

        # Copy the tarballs to /tmp as me, and unpack as root later.
        # (Could get them via Samba, except Root has no Samba access.)
        v -L $updates/*_$date.tgz
        foreach node ( $nodes )
          echo ================  $node.$exp
          scp -p $updates/*_$date.tgz $node.$exp\:/tmp
        end

        # First update Cygwin setup, then run it to update Cygwin itself.
        chmod 755 /share/windows/cygwin-setup.exe
        nodes "cp -p /share/windows/cygwin-setup.exe /tmp/setup.exe"
        su_nodes "(cd C:/Software/Cygwin; chown -R root.None .; chmod g+w ./setup.exe)"
        su_nodes "(cd C:/Software/Cygwin; cp /tmp/setup.exe .; rm -f ./cygwin-setup.exe; ls -l)"

          # Log in a bunch of RDP windows, *as root*.
          set rootpwd='rootPwd'
          set rd=/proj/testbed/fish/rdesktop/rdesktop-1.3.1
          set rdarg="-K -g 1280x1024 -a 16"
          echo $nodes $exp
          foreach node ( $nodes )
            echo $node.$exp
            (cd $rd; ./rdesktop $rdarg -u root -p "$rootpwd" $node.$exp &)
            sleep 1
          end

            #[On the node, as root, under RDP.]
            echo $HOME
            regtool get /HKLM/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/SetHOME

              # First update just the cygwin package so post-install scripts work.
              # Click the Keep radio button, then click the New column to the new
              # version for the Base/cygwin package.  Probably have to reboot after.
              /cygdrive/c/software/cygwin/setup.exe &
              /sbin/reboot

              # Gotta restart the RDP windows; but of course the following can't be done under ssh.
              foreach node ( $nodes )
                echo $node.$exp
                (cd $rd; ./rdesktop $rdarg -u root -p "$rootpwd" $node.$exp &)
                sleep 1
              end

              # Installing OpenSSH hangs if you update it while it's running.
              cygrunsrv -VQ sshd
              cygcheck -c openssh
                ls -l /etc/moduli
                ps -Welf | grep ssh
              net stop sshd
                killall ssh-agent

              # Ditto agetty.  The SysV init daemon keeps it running.
                ps -Welf | grep 'agetty\|init'
                cygrunsrv.exe -VQ init
              net stop init
                ps -Welf | grep 'agetty\|init'
                cygrunsrv.exe -VQ init
              mv /sbin/agetty.exe{,.save}

                # May have to actually uninstall OpenSSH.  Click the Keep button, then
                # click the New entry to Uninstall on net/openssh.
                /cygdrive/c/software/cygwin/setup.exe &
                # Re-install openssh again.
                /cygdrive/c/software/cygwin/setup.exe &

            # Now update everything else.
            /cygdrive/c/software/cygwin/setup.exe &
              # May need to reboot and run again before adding any new packages like sysvinit.
              cygcheck -c openssh
              cygcheck -c sysvinit
              cygcheck -c psmisc
              /sbin/reboot
              # Restart the RDP windows again.
              foreach node ( $nodes )
                echo $node.$exp
                (cd $rd; ./rdesktop $rdarg -u root -p "$rootpwd" $node.$exp &)
                sleep 1
              end

            # Unpack the patched and edited system files.
            date=2006-11-16

            # Includes agetty.exe, under the SysV /etc/inittab.
            # (Don't stop init in a console window, it kills agetty, and hence your login.)
            net stop init
            chmod 777 /etc/inittab
            tar xvfz /tmp/sys_$date.tgz -C /
              ls -l /sbin/agetty*
            chmod 600 /etc/inittab
            net start init

            # sshd.exe may be busy, move the busy executable aside to install it.
              ls -l /usr/sbin/sshd*
            cd /usr/sbin
            net stop sshd
            mv sshd.exe{,.prev}
            mv sshd.exe{.new,}
              ls -l /usr/sbin/sshd*
            net start sshd

          # Now that Cygwin and sshd are updated, we can go back to scripting...
          #[On Boss.]
          # Shut down the Emulab services so we can install new programs.
          nodes "ps -Welf | grep emulab"
          nodes "killall evproxy"
          su_nodes "/usr/local/etc/emulab/rc/rc.syncserver shutdown; \
                    /usr/local/etc/emulab/rc/rc.progagent shutdown; \
                    /usr/local/etc/emulab/rc/rc.slothd stop; \
                    /usr/local/etc/emulab/rc/rc.linktest stop; \
                    ps -Welf | grep emulab"
          # Just in case... (Try it again if something is still running afterward.)
          su_nodes "killall emulab-sync program-agent slothd idlemon linktest; \
                    ps -Welf | grep emulab"
          # Go into Task Manager and kill processes (probably idlemon) if nothing else worked.
          nodes "ps -Welf | grep emulab"

          # Unpack the Emulab files.
          eval 'su_nodes "'" tar xfoz /tmp/elab_$date.tgz -C /usr/local "'" '
          nodes "ls -lt /usr/local/etc/emulab | head -5"

     . Do one-time updates
         #[On Boss.]
         su_nodes "bootcfg /query; bootcfg /ems OFF /id 1; bootcfg /query"
         su_nodes "chown system.system /etc/inittab /etc/rc; \
                  cygrunsrv -I init -d 'Cygwin SysV init' -p /sbin/init -a -i -s INT; \
                  net start init"

         # Install idlemon user-login wrapper so that it starts up hidden.
         # It touches /var/run/rdp_input for slothd to see.
         # idlemon.vbs is in the registry on SP0 and SP1, but fails to start idlemon.
         # idlemon.exe runs okay on all versions.
         ##su_nodes "regtool -s set /HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/EmulabIdle 'C:\cygwin\usr\local\etc\emulab\idlemon.vbs'"
         ## The su_nodes alias just isn't up to handling complicated quoting...
         foreach node ( $nodes )
           echo ================  $node.$exp
           sudo ssh -n $node.$exp "regtool -s set /HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/EmulabIdle '"'C:\cygwin\usr\local\etc\emulab\idlemon.vbs'"'"
           echo ""
         end
         su_nodes "regtool get /HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Run/EmulabIdle"

         # 11/22/05 Update rxvt and Cygwin shell links.
         set audtop=/cygdrive/c/Documents\ and\ Settings/All\ Users/Desktop
         foreach node ( $nodes )
           echo ================  $node.$exp
           sudo scp -rp /share/windows/Desktop $node.$exp":"/tmp
           sudo ssh -n $node.$exp "chown root '$audtop'"
           sudo ssh -n $node.$exp "cp /tmp/Desktop/*Shell.lnk '$audtop'"
           sudo ssh -n $node.$exp "chown -R SYSTEM '$audtop'"
           sudo ssh -n $node.$exp "ls -l '$audtop'"
           echo ""
         end

         # 2/3/06 Update rude/crude for linktest, do a little cleanup.
         nodes "ls -lt /usr/local/bin | head"
         v /share/windows/{c,}rude.exe
         foreach node ( $nodes )
           echo ================  $node.$exp
           sudo scp -rp /share/windows/{c,}rude.exe $node.$exp":"/usr/local/bin
         end
         nodes "chmod 755 /usr/local/bin/{c,}rude.exe"
         su_nodes "ls -l /usr/local/etc/emulab/{*.exe.prev*,*~}"
         su_nodes "ls -l /usr/local/etc/emulab/rc/*~"
         su_nodes "rm /usr/local/etc/emulab/rc/*~"
         su_nodes "rm /usr/local/etc/emulab/rc/rc.cygwinxp-user"
         # Clear Administrator desktop; shortcuts are copied to all Administrators group users.
         su_nodes "ls /cygdrive/c/Documents\ and\ Settings"
         # (There may be none if there has been no login as Administrator...)
         su_nodes "ls /cygdrive/c/Documents\ and\ Settings/Administrator/Desktop"
         su_nodes "rm /cygdrive/c/Documents\ and\ Settings/Administrator/Desktop/*.lnk"

         # 2/10/06 Make sure autoconfiguration is turned off.
         set svcs=/HKLM/SYSTEM/CurrentControlSet/Services; set tcpp=$svcs/Tcpip/Parameters
         foreach node ( $nodes )
           echo ================  $node.$exp
           ssh -n $node.$exp "regtool get $tcpp/IPAutoconfigurationEnabled"
         end
         foreach node ( $nodes )
           echo ================  $node.$exp
           ssh -n $node.$exp "regtool set -i $tcpp/IPAutoconfigurationEnabled 0"
         end

         # 7/28/06 The rude/crude executables are now patched in emulab/bin.
         su_nodes "mv /usr/local/bin/rude.exe{,.save}"
         su_nodes "mv /usr/local/bin/crude.exe{,.save}"

       - Some things still have to be done through the GUI under RDP on each node.

         . **** Run Windows Update (Express) after each Patch Tuesday on the UPDATE nodes. ****
           It will probably want to reboot at least once.

         . Go into Control Panel / Power Options / Hibernate, and make sure it's disabled.
           Also set Power Options / Power Schemes to "Always On".

         . 10/28/05 Install the Cygwin ping package, keep everything else the same.
           /cygdrive/c/software/cygwin/setup.exe &

         . 11/22/05 Install the Cygwin rxvt package and back-rev OpenSSH from 4.2 to 4.1 .
           /cygdrive/c/software/cygwin/setup.exe &

         . 03/20/06 Set DriversPath and right-click/Install the drivers/nic .inf files
           in Windows Explorer to put them into the Drivers DB.
               regtool get /HKLM/SOFTWARE/Microsoft/Windows/CurrentVersion/DevicePath
             regtool set -s /HKLM/SOFTWARE/Microsoft/Windows/CurrentVersion/DevicePath \
                                'C:\WINDOWS\inf;C:\drivers\disk;C:\drivers\nic'
               v C:/drivers
             sysprep=/share/windows/sysprep
             cp -rp $sysprep/drivers /cygdrive/c
             # You can select them all in /cygdrive/c/drivers/nic and right-click/Install once.
             # It makes corresponding PNF files in the directory and remembers DevicePath.
             # Tip: view Details, sort by type to list the "Setup Information" files together.
             e1000325.inf
             e100a325.inf
             e100b325.inf
             netag311.INF

     . Prepare, reboot and test.
         #[On Boss.]
           su_nodes "ls -l /var/log"
           su_nodes "ls -lt /var/log"
         # Make sure there aren't USER logins blocking their profiles from removal. (Root is okay.)
         nodes qwinsta

         # (Insert root password for "rootPwd" below.)
         ##su_nodes "prepare -p rootPwd"
         ## Have to do "prepare -n -s" on the SERIAL CONSOLE.  It shuts down the network.
         ##
         ## On SP0, expect "Error (2): The system cannot find the file specified."
         ##
         ## The following is also normal, although the directory winds up empty at the end:
         ##   "rm: cannot remove directory `/var/emulab/boot/tmcc': Directory not empty"
         ##
         prepare -n -s
         cat /etc/motd

         ### Checking cycle before making images. ###
         # It won't have to do the double-reboot dance, because the node names stay the same.
         # But, if you use SysPrep (-s option) it has to reboot after reconfiguring everything...
         su_nodes "/sbin/reboot"
         ##node_reboot -e $pid,$eid
           ## node_reboot -e is broken.
           ## It reboots the firewall too, then the nodes time out on DHCP in PXE boot.
           ## Do it by hand on just the Windows nodes in the experiment for now.
           ## Translate $node.$exp to pcXXX to avoid "*** Tainted node name: ".
           foreach node ( $nodes )
             echo ================================ $node.$exp
             set bootpc=`nslookup $node.$exp | sed -n '/^Name:/s/.*\(pc[0-9]*\).*/\1/p'`
             node_reboot -f $bootpc
           end
         su_nodes "ps -Welf | grep emu"

         # Network configs.
         su_nodes "ipconfig /all"
         su_nodes "grep number /var/log/bootsetup.log"
         su_nodes "grep -A 4 Fixing /var/log/bootsetup.log"
         # Should be set to 1 on the SP2 node in the topology, 0 elsewhere.
         su_nodes "grep IPEnableRouter /var/log/EmulabStartup.log"

             # Test network setup.
             ##Oops, firewall doesn't do linktest, so this fails.
             ssh ops run_linktest.pl -v -e $pid/$eid

             ## set nodes="node1 node2 node3 node4 node5"
             ## set exp=windows-5.testbed.emulab.net
             ## set exp=windows-5a.testbed.emulab.net
             ## set exp=test-windows-pc600.testbed.emulab.net
             ## set exp=test-windows-pc850b.testbed.emulab.net
             ## set exp=test-windows-pc3000.testbed.emulab.net
             ## set pid=testbed eid=test-windows-pc3000

         # Simple connectivity test.
         foreach node1 ( $nodes )
           foreach node2 ( $nodes )
             if ( "$node1" != "$node2" ) then
               echo ================================================================
               echo $node1 pings $node2
               sudo ssh $node1.$exp "ping -n 1 $node2" | grep loss
             endif
           end
         end
         # Linktest.
         run_linktest.pl -v -l 4 -e $pid/$eid
         run_linktest.pl -v -l 2 -e $pid/$eid
         run_linktest.pl -v -l 3 -e $pid/$eid

             # Tiny script updates (transmit from current directory.)
             set fixfile=liblocsetup.pm fixdest=/usr/local/etc/emulab/
             foreach node ( $nodes )
               echo ================================ $node.$exp
               scp -p $fixfile $node.$exp":"/tmp
             end
             eval 'su_nodes "'" diff $fixdest/$fixfile /tmp/$fixfile "'" '
             eval 'su_nodes "'" cp /tmp/$fixfile $fixdest/$fixfile "'" '
             eval 'su_nodes "'" ls -l $fixdest/$fixfile "'" '
             eval 'su_nodes "'" chown root $fixdest/$fixfile; chmod 755 $fixdest/$fixfile "'" '

         # Check the Windows version.  "Help/About Windows" is "winver", no cut-and-paste.
         set cv="/HKLM/SOFTWARE/Microsoft/Windows NT/CurrentVersion"
         # There's no service pack level for SP0 - Don't worry about the error.
         foreach node ( $nodes )
           echo -n "$node - "
           ssh $node.$exp 'echo "`regtool get '" '$cv/ProductName'" '`'\
                          '(Build `regtool get '" '$cv/BuildLab'" '`'\
                          ': `regtool get '" '$cv/CSDVersion'" '`)" '
         end

     . Prepare again, go into admin mode, and pull the images.
         # (Insert root password for "rootPwd" below.)
         ##su_nodes "prepare -p rootPwd"
         ## Have to do "prepare -n -s" on the serial console.
         prepare -n -s
           #[On a single Windows Update node, set the motd line to UPDATE.]
           cat /etc/motd
           ed /etc/motd
           1
           s/node/UPDATE/p
           w
           q

         ##node_admin -e $pid,$eid on
           ## node_admin -e is broken.
           ## It reboots the firewall too, then the nodes time out on DHCP in PXE boot.
           ## Do it by hand on just the Windows nodes in the experiment for now.
           ## Translate $node.$exp to pcXXX to avoid "*** Tainted node name: ".
           foreach node ( $nodes )
             echo ================================ $node.$exp
             set pc=`nslookup $node.$exp | sed -n '/^Name:/s/.*\(pc[0-9]*\).*/\1/p'`
             node_admin on $pc
           end

           # Check that they're booted into the FreeBSD MFS.
           # (FreeBSD 5.3-STABLE (TESTBED) #1: Thu Oct 21 15:06:28 MDT 2004)
           foreach node ( $nodes )
             echo ================================ $node.$exp
             sudo ssh $node.$exp cat /etc/motd | head -1
           end

         nping
         echo $date
         v -t /{usr,proj}/testbed/images/WINXP-*$date*.ndz
           v -t /{usr,proj}/testbed/images/WINXP*.ndz | head
         df -m /{usr,proj}/testbed/images

             # One at a time...
             set node=SP0
             set node=SP1
             set node=SP2
             set node=UPDATE
             set imagefile=/proj/testbed/images/WINXP-$node-$img"_"$date.ndz
         # All together.
           echo $hw $img $disk $date
         foreach node ( $nodes )
           echo ================================  $node.$exp
           set imagefile=/proj/testbed/images/WINXP-$node-$img"_"$date.ndz
           echo $hw $img $disk $imagefile
           sudo ssh -n $node.$exp "imagezip -o -I 2 -I 3 -I 4 $disk $imagefile"
           echo ""
         end

         ##node_admin -e $pid,$eid off
           foreach node ( $nodes )
             echo ================================ $node.$exp
             set pc=`nslookup $node.$exp | sed -n '/^Name:/s/.*\(pc[0-9]*\).*/\1/p'`
             node_admin off $pc
           end

               # Reload the os image on each node.  Node name matches the image name.
               foreach node ( $nodes )
                 echo ================================ $node $pid,$eid
                 set pc=`nslookup $node.$exp | sed -n '/^Name:/s/.*\(pc[0-9]*\).*/\1/p'`
                 os_load -p $pid -i WINXP-$node-new $pc &
               end


     . Move the images to /usr/testbed/images to avoid NFS reads, for faster swap-in.
        df -m /{proj,usr}/testbed/images
          v /{proj,usr}/testbed/images/WINXP-*-*_$date.ndz
        set projimages=/proj/testbed/images/WINXP-*-$img"_"$date.ndz
          v $projimages

            ##cp -p $projimages /usr/testbed/images
        ## NFS gets swamped on Ops, use scp instead of cp.
        foreach prjimg ( $projimages )
          v $prjimg
          set usrimg=/usr/testbed/images/$prjimg:t
          echo $prjimg $usrimg
          scp -p ops:$prjimg $usrimg.new
          # Install with mv.  Frisbee might have the old inode still open.
          if ( -f $usrimg.prev ) rm -f $usrimg.prev
          if ( -f $usrimg ) mv $usrimg{,.prev}
          mv $usrimg{.new,}
          v $usrimg
          if ( -f $usrimg.prev ) rm -f $usrimg.prev
        end
        # Check that everything arrived correctly.
          ssh -n ops cksum $prjimg & cksum $usrimg &
        ssh -n ops cksum $projimages > ~/projimages.cksums &
        set usrimages=/usr/testbed/images/WINXP-*-$img"_"$date.ndz
        cksum $usrimages > ~/usrimages.cksums &
          jobs
          wait
          diff ~/projimages.cksums ~/usrimages.cksums
        sed 's/proj/usr/' ~/projimages.cksums | diff - ~/usrimages.cksums

        # The ones on Ops:/proj are not needed anymore.
        v $usrimages
        v $projimages
        rm -f $projimages

     . Make symlinks without the date suffix, corresponding to the Image ID's.
         # First, -new symlinks for testing.
         cd /usr/testbed/images
         set pctype; if ( "$hw" != pc850 ) set pctype="-$hw"
         set imgs="{SP[012],UPDATE}"
           echo $node
           # set imgs=$node
         ls -lt WINXP-$imgs$pctype-new.ndz
         ls -lt WINXP-$imgs-$img"_"$date.ndz
         foreach node ( $nodes )
           set link=WINXP-$node$pctype-new.ndz file=WINXP-$node-$img"_"$date.ndz
           echo "$link -> $file"
           if ( -l $link ) rm -f $link
           ln -s $file $link
         end
         v WINXP-$imgs$pctype-new.ndz
         v -L WINXP-$imgs$pctype-new.ndz
           # Upward compatability from -pc3000 images to -sysprep images.
           set hw=pc3000 img=sysprep pctype=-pc3000
           # Restore.
           set hw=pc850 img=sysprep disk=/dev/ad0 pctype=

         # Update the dates in the image ID's.
         ls -lLT WINXP-$imgs$pctype-new.ndz
         ls -lLT WINXP-$imgs$pctype-new.ndz | gawk '{ sq = "'"'"'"; \
                # Convert from "Oct 1 18:24:27 2005" to "2005-10-01 18:24:27" date format. \
                "date -j -f \"%b %d %T %Y\" \""$6" "$7" "$8" "$9"\" +\"%F %T\"" | getline date; \
                image = gensub("\.ndz", "", 1, $NF); \
                image = gensub("UPDATE-pc3000-new", "UPD-pc3000-new", 1, image); # Special case.\
                print "update images set created=" sq date sq " where imagename=" sq image sq ";"\
                print "update os_info set created=" sq date sq " where osname=" sq image sq ";" }' \
                | tee ~/imgdates.sql

         echo "select created, imagename, path, pid from images where imagename like 'WINXP-%$pctype-new';" | mysql tbdb | sort
         echo "select created, osname, pid from os_info where osname like 'WINXP-%$pctype-new';" | mysql tbdb | sort

         mysql tbdb < ~/imgdates.sql
         echo "select created, imagename, path, pid from images where imagename like 'WINXP-%$pctype-new';" | mysql tbdb | sort
         echo "select created, osname, pid from os_info where osname like 'WINXP-%$pctype-new';" | mysql tbdb | sort

         # Check before installing the non -new versions.  (WINXP will always be in partition 1.)
         echo "select part1_osid from images where imageid like 'emulab-ops-WINXP%';" | mysql tbdb

         # Then the regular symlinks.
         cd /usr/testbed/images
         set pctype; if ( "$hw" != pc850 ) set pctype="-$hw"
         v WINXP-$imgs$pctype.ndz
         v WINXP-$imgs-$img"_"$date.ndz
         foreach node ( $nodes )
           set link=WINXP-$node$pctype.ndz file=WINXP-$node-$img"_"$date.ndz
           echo "$link -> $file"
           if ( -l $link ) rm -f $link
           ln -s $file $link
         end
         v WINXP-$imgs$pctype.ndz
         v -L WINXP-$imgs$pctype.ndz
           # Upward compatability from -pc3000 images to -sysprep images.
           set hw=pc3000 img=sysprep pctype=-pc3000
           # Restore.
           set hw=pc850 img=sysprep disk=/dev/ad0 pctype=

         # Update the dates in the image ID's.
         ls -lLT WINXP-$imgs$pctype.ndz
         ls -lLT WINXP-$imgs$pctype.ndz | gawk '{ sq = "'"'"'"; \
                # Convert from "Oct 1 18:24:27 2005" to "2005-10-01 18:24:27" date format. \
                "date -j -f \"%b %d %T %Y\" \""$6" "$7" "$8" "$9"\" +\"%F %T\"" | getline date; \
                image = gensub("\.ndz", "", 1, $NF); \
                print "update images set created=" sq date sq " where imagename=" sq image sq ";"\
                print "update os_info set created=" sq date sq " where osname=" sq image sq ";" }' \
                | tee ~/imgdates.sql
         echo "select created, imagename, path, pid from images where imagename like 'WINXP-%$pctype';" | mysql tbdb | sort
         echo "select created, osname, pid from os_info where osname like 'WINXP-%$pctype';" | mysql tbdb | sort

         mysql tbdb < ~/imgdates.sql
         echo "select created, imagename, path, pid from images where imagename like 'WINXP-%$pctype';" | mysql tbdb | sort
         echo "select created, osname, pid from os_info where osname like 'WINXP-%$pctype';" | mysql tbdb | sort
