NMAP

This is a perl script that shows what perl does well. It takes a couple of programs which provide useful output -- nmap and nbtstat -- and combines them into a single daily archive showing what's on your network. Key features are:

• The input is a list of named IP ranges.
• The output is written to hosts.txt and yyyy-mm-dd.hosts.txt automatically building an archive of what's up in the networks you care about.

To run it

1. Install nmap and make sure it's in the path. nbtstat is part of any Windows installation.
2. Install Perl, which means Active State.
3. Create a range file that's just a text file with a few lines like these: (Obviously with your network ranges. You can use any range specification that nmap will accept.)

     # my range.txt 
   servers    192.168.10.0/24 
   perimeter  192.168.12.0/24 

4. Make sure your reverse DNS is working, or you won't get any DNS names.
5. Run it with a command like c:\perl\bin\perl nmap.pl range.txt
6. check the output in hosts.txt to see server names, and, for windows servers, domain and server names.

To get the benefit, set it to run every day using a scheduled task (I find this is easiest with a .CMD file containing full path names for everything) and in a few months you'll have some worthwhile history.

# Start of nmap.pl by umacf24
# This program takes a single parameter -- a file containing  pairs.
# Example (but leave off the #)
# servers    192.168.10.0/24
# perimeter 192.168.12.0/24
#
use strict;
use warnings;

# NBTSTAT codes from Jim Halfpenny
my %group  = (  hex("00"), "00 Dom",
  hex("01"), "01 M Browser",
  hex("1C"), "1C Domain Controller",
  hex("1E"), "1E S Browser", # Browser Elections
);
my %unique = (  hex("00"), "00 WS",    # Workstation service
  hex("01"), "01 Msgr", # Messenger Service
  hex("03"), "03 Msgr", # Messenger Service
  hex("06"), "06 RAS Server Service",
  hex("1B"), "1B Domain Master Browser",
  hex("1D"), "1D Master Browser",
  hex("1F"), "1F NetDDE Service",
  hex("20"), "20 SVR", # File Server Service
  hex("21"), "21 RAS Client Service",
  hex("22"), "22 MS EXC Interchange(MSMail Connector)",
  hex("23"), "23 MS EXC Store",
  hex("24"), "24 MS EXC Directory",
  hex("30"), "30 Modem Sharing Server Service",
  hex("31"), "31 Modem Sharing Client Service",
  hex("43"), "43 SMS Clients Remote Control",
  hex("44"), "44 SMS Administrators Remote Control Tool",
  hex("45"), "45 SMS Clients Remote Chat",
  hex("46"), "46 SMS Clients Remote Transfer",
  hex("4C"), "4C DEC Pathworks TCPIP service on Windows NT",
  hex("42"), "42 McAfee AV",
  hex("52"), "52 DEC Pathworks TCPIP service on Windows NT",
  hex("87"), "87 MS EXC MTA",
  hex("6A"), "6A MS EXC IMC",
  hex("BE"), "BE Netmon Agent",
  hex("BF"), "BF Netmon Application",
);

# results stored in an array for re-use
my @nmapout;

my $ofn='hosts.txt';
my $opath='.\\ip\\';

my $header = "Key\t++ Up but unnamed, \n\t** Named but not responding to ping.\n\t?? Improper NBT names\n";
$header .= "Ping scan run between ".localtime(time())." ";

my $ipall; my $ipup; my $ipdown; # Address counters

# Process the file of network names. Run NMAP against the network spec (iprange).
while (<>)
{
  next if (/^\s*#/) ;
  my @line = split /\s+/;
  if (2==@line)
  {
      my $ipname = sprintf "%-15.15s ", $line[0]; # Fixed width
      my $iprange = $line[1];      

      my $cmd = ".\\bin\\nmap -sP -R -oG - $iprange |";    # local nmap
      print STDERR "$cmd\n";
    
      open (NMAP, $cmd);
      while ()
      {
          chomp;
          next if /Smurf/;
          if (/^Host: /)
          {
              $ipall++;
              my $hostline=$_;
              $ipdown++ if ($hostline =~ /Status: Down$/);
              $ipup++ if ($hostline =~ /Status: Up$/);
              my $flag ='   ';
              $flag = '++ ' if ($hostline =~ /\(\)\tStatus: Up/) ;# Un-named
              $flag = '** ' if ($hostline =~ /\([^)]+\)\tStatus: Down/) ;# Named, but not responding
              my $hostip = $1 if ($hostline =~ /^Host: ([0-9\.]+) .+ Up$/);
              my $prefix=$ipname . $flag;
              my $postfix=nbtstat($hostip) if ($hostip);
              $hostline =~ s/^Host: /$prefix/;
              $hostline =~ s/Status: Up$/NBT: $postfix/ if ($postfix);
              # print STDERR "Host Line $hostline | $postfix | $hostip\n";
              push @nmapout, $hostline;
          }
      }
  }
}
$header .= "and ".localtime(time())."\n";

if ($ipall)
{    # Some addresses seen
  $header .= "$ipall addressses pinged:- Up: $ipup, Down: $ipdown\n";
}
else
{
  $header .= "No addresses scanned. NMAP in path?\n";
}

# Preserve an archive Version
my ($sec, $min, $hr, $day, $mon, $yr, $wday, $yday, $isdst) = localtime(time());
$yr = $yr + 1900;
$mon = $mon + 1;
my $prefix = sprintf "%04d-%02d-%02d.", $yr, $mon, $day ;
my $afn="$opath$prefix$ofn";
my $hfn="$opath$ofn";

#Don't overwrite existing hosts file until we have finished scanning.
open (AHOSTS, ">$afn") or die "could not open $afn $!";
open (HOSTS, ">$hfn") or die "could not open $hfn $!";
print AHOSTS $header; print HOSTS $header;
foreach my $hostline (@nmapout)
{
  print HOSTS "$hostline\n";print AHOSTS "$hostline\n";
}
close HOSTS; close AHOSTS;

sub nbtstat
{
  my ($hostip) = @_;
  my $cmd = "nbtstat -a $hostip |";
  my %survey; my $ret;
  my $domain; my $server;

  # print STDERR "survey $cmd\n";
  open (NBTSTAT, $cmd) or die "can't run $cmd $!";
  while ()
  {
      # print STDERR;
      # The meat of the NBTNAME command is in this format: name  type
      if (/^\s+(\S+)\s*<([0-9A-F]{2})>\s+(GROUP|UNIQUE)/i )
      {
          my $nbtname = $1;
          my $nbtclass = $2;
          my $nbttype = $3;
          if ('00' eq $nbtclass)
          {
              $domain = $nbtname if ('GROUP' eq $nbttype);
              $server = $nbtname if (('UNIQUE' eq $nbttype) && ($nbtname !~ /(\~)|(MSBROWSE)/));
          }
          my $desc = ('GROUP' eq $nbttype)? $group{hex($nbtclass)} : $unique{hex($nbtclass)};
          $desc = "$nbtclass $nbttype Unknown" unless $desc;
          # print STDERR "$nbtname $nbtclass $nbttype $desc \n";
          push (@{$survey{$nbtname}}, $desc);
      }
  }
  close NBTSTAT;
  # We should have a good domain name and a good server if we have anything
  if (%survey)
  {
      $ret = ($domain and $server) ?  "$domain\\$server" : '??';
  }
  foreach my $nbtname (keys %survey)
  {
      $ret .= " $nbtname: ("  . join (', ', @{$survey{$nbtname}} ). ")" unless ($nbtname =~ /(\~)|(MSBROWSE)/) ;
  }
  return $ret;
}
# End of nmap.pl

Swirl/Unswirl

Here's the Interpol (rare to see a .int domain) release on operation Vico. That's impressive work by the image experts, but I think the credit lies with the investigator who had the imagination to see that information was still there and the thing was worth trying. That's the leap I would have failed to take, and it's the difference between a proper investigator and the guy who looks at proxy logs.

After red-eye reduction, obscuring faces is probably the second most popular use for home image processing, the home porners generally botch it, and I've often wondered why packages don't include a better tool. Perhaps they can't find a suitably euphemistic name. Anyway, I suppose we'll now see a slew of unswirl tools.

The Approver as a Conceptual Bottleneck

I'm looking at a list of domains groups which control access to removable devices and media through Pointsec Protector (which used to be Reflex Magnetics Disknet Pro). We've had groups for various types of devices and now I'm trying to simplify -- to operate a much cruder level of control.

I'd prefer to leave it as it is, but the membership of the current groups is a mess. The technology is fine, but the control environment stinks. At present we allow or deny access to the groups based on a managers approval: "he has a business need to use a USB key" -- and that makes a good deal of sense. Who else can make that choice?

Who else indeed? Because the managers aren't technical -- so they don't understand what they're approving -- and they don't see any downside from insecure access. Essentially every request gets approved. And in six months time when the the lists have to be recertified, it's probably a different manager, or the original justification is forgotten, and it's easier just to agree. We've got adequate technology, and a process that the auditors think is just fine, and there's no real control at all.

Of course, this isn't just a problem for USB devices. It very easy to fail at this last hurdle by asking approvers to use a discretion that they just can't understand.

I've thought about making these accesses part of the permissions attached to the job description global groups. But that doesn't reduce the problem unless IT security can engage with the role definition approvers, and we don't.

So this is my plan: I'm going to name the groups after the risk, with alarming group names and descriptions:
Risk In -- "Trusted to read data from unknown sources"
Risk Out -- "Trusted to send corporate data to unknown destinations"
Device Risk -- "Trusted to attach untrusted and untested devices"

Let's see whether that gets the message across.

War Against the Aliens

In Arthur C. Clarke's Rendevous With Rama, the world of the 23rd century prepares for a visit by potentially hostile aliens. A minor character, a general, justifies military preparations by claiming that even though humans have complete dominion over a wasps nest, we still leave it alone if we possibly can. So even though the aliens may be vastly more powerful than we are, they would still care enough about nuclear missiles to leave us alone.

I suppose that's comforting. But it's not very comforting. The wasps nest I destroyed on Sunday afternoon had little idea who I was, and no idea why I was attacking. Alas! they didn't know I was afraid of their stings and I would have left them alone if only they weren't so dangerous. Even worse, the means chosen by the superior alien -- me -- was an organophosphorus agent: nerve gas. We might be better off without our stings.

Absolutely the Last Apple Entry this Year

It's been an astonishing year for fruit, but there's a price to pay. In ordinarily heavy years, plum tree branches can easily break with the weight of fruit. This year has had so much water that the damsons were the size of plums and edible straight off the tree while the table plums have needed props, to hold up the masses of disappointing bland fruit. And while the flavours been fine, we've had apple branches broken to create more challenges for the winter pruner. I drove past this old orchard today and saw trees pulled over or split in two -- it'll take more than pruning to fix that.

Proxy Access for Services

Of course you want to use a web proxy, but some of your services need web access. Proxy settings are per-user, and if you run services as specific users you can log on and set them. But for the built-in anonymous accounts SYSTEM, SERVICE, how can you tell them where to find the proxies?

The obvious need for this is to get Windows Update working behind a proxy server. It's needed even if you are using the web interface, because WU still depends on the BITS service.

Well there are a number of ways. But what's easy is proxycfg, a command-line program that will create the appropriate entries in

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\WinHttpSettings

The program is in the XP build, but it runs fine on W2K -- just copy it over. Running it with the -u option will copy the current user's settings in to the service default and you're done.

Of course, you still need to ensure that the requests will be permitted by the proxy: The service can't authenticate. On our Bluecoats, you make a combined destination object that precedes authentication and is accepted on the first rule.

Cider

Last weekend I picked a lot of apples: 50% Bramley, 30% James Grieve, 20% Spartan with oddments of John Downie crabs and Conference pear, and screwed it down to a bit over four gallons of juice. I'd say there was a ratio about three gallons of loose apples to one of juice, but that may be optimistic, They were all fresh off the tree and hard, so I don't think I mashed them as well as I should have. Anyway, after a rather slow start, it's fermenting nicely with whatever Wilkinson sell as wine yeast at this time of year.

It was extremely hard work. The book had been rather lyrical about the benefits of community endeavour with everybody helping to get the work done. Mrs U left me to it, driving off to visit her parents, and I pounded out the whole lot using the boss on a 17lb fence post digger and a six liter screw press myself.

Since I set that lot going, I've learnt that 80% cookers is a bad thing (too acid), that using fresh apples reduces the juice yield (too hard, and too tart), and basically I've done it all wrong. So today I filled the mower trailer with 60% eaters and the balance mostly Bramleys and I've hidden it in the shed. In a fortnight's time I'll see whether they've softened up, and perhaps make myself some less acid cider to blend. For certain the Spartans had the dullest juice so I'm afraid the cider will be bland, but we'll have to see how that goes.

Idle Sods

I've just been through my spam, and I'm disgusted. Where are all the Northern Rock "After the difficulties on our website, please re-enter your details to restore your access..." or "The bank has been nationalised. Please provide details to validate access to your deposit...." ?

Don't these people read the news?

I know a secret

And it's a surprise: the best juice comes from crab apples.

Admittedly they were ripe red John Downie, but straight off the tree they were still a lot more like hard red berries than proper apples, and they were the devil to crush. I got three pints from a gallon of mush, and it was sweet and appley with a lot of the puckering richness which I think is malic acid. Drinkable - delicious - running off the press.

Freedom

Anyway, yes Holiday time in the very wonderful Isle of Wight. If you've got odd ASD children, you probably should be booking about now for one of next years Freedom Family week at East Dene. Why? a) Ventnor is a smashing little seaside resort. If you like walks, there are lovely walks there and back, or to Shanklin. b) The IoW railway will engage any train fiend, especially if you go all the way to the end of the pier, and it connects with steam trains if that's your bag. c) If you prefer to do nothing, the outdoor pool is heated, attended and open for hours a day. Most ASD kiddywinks and their siblings will play happily and tiringly in the pool for hours, and if they don't want that, there are activities most days. d) The air and views are conducive to a feeling of irrepressible well-being. e) It's catered. If you've been driven to self catering just to avoid the odd looks, it's amazingly wonderful not to have to hit the supermarket, look for pepper or whatever.

But none of that matters, because the real colossal advantage is is that however weird, badly behaved or just plain crazy your kiddies are, other children are crazier. Howls like a gibbon in the swimming pool? No problem. Unreliable judgement on wearing trousers? Don't we all. Turds on the staircarpet? Got to do it somewhere. True for all the other guests, true for staff. Amazingly relaxing, even if you're sleeping in a bunk and eating canteen food.

As a bonus you get to meet all the other odd children. Other people's kiddies are always more fun (because you get to hand them back) but this year I had the odd sensation that I was meeting children who were amazing. That's not a conventional wishy-washy feelgood disabled rights amazing, but absolutely fuck-off precocity and ability. Some were diagnosed, and some were slightly dodgy "we think he's all right" siblings. Amazing. I wish I could publish a list of names to watch out for because some of those kiddies are going far.......

Physical Security

I would have said there was nowhere in the UK, certainly in England, where you could run rocket engines for an orbital lifter without it being obvious to everyone around. It's all too small, and rockets are so very, very obvious.

Sucks to me then, because from the late 50's up until 1972, Saunders-Roe in Cowes managed strict secrecy while test firing assembled Black Knight and Black Arrow rockets right next to a tourist attraction on the Isle of Wight! Tucked away in a fold of Tennyson Down above the Needles Battery was a complete static test rig, with exhaust and cooling steam directed out to sea high above the cliffs. Less than a mile from the archetypical 50's tourist resort of Alum Bay, they kept their secret with nothing more than a wire fence, MoD police and a slightly deceptive line of sight.

Actually, there's one other thing you can only appreciate if you visit the site. The fantastic wind may not be louder than a rocket engine, but it would disperse it utterly.

This is all very fine, and the exhibition in the chalk bunkers of the new battery is well worth a look. But you have to be a real nerd to know that in Moonraker -- published in 1955 -- Ian Fleming had Hugo Drax, his nazi revanchiste, building and launching the rockets to destroy London from silos in the chalk cliffs between Dover and Folkstone. Fleming's intelligence contacts were the best: he must have known. Did the S-R security officer have kittens? Were there hard words exchanged? Was James Bond, of all people, responsible for a breach in national security?

Don't fear the Domain

This month has been packed with Windows servers not wanting to go in the Windows domain. Telephony servers, firewall management. And every time, the reason given is: it's more secure.

This reason is bollocks.

I can see the psychological justification: it feels like you are mixing your security sensitive servers with the common run. There is a sensation of "taint". But every time, it comes down to this: Domain membership lets domain admins do what they want with it.

The simple answer to this is that if you don't trust your domain admins you have your domain management wrong. And, more importantly, if you have more than one admin, the weakening of control caused by the shared user that will be used on the non-domain system substantially outweighs the possibility of a rogue assigning rights improperly. Let's be clear:

Not a Domain Member = Anonymous User IDs

Do you still want to be out of the domain. Really, what's possibly worse or less controlled than a shared user? Nothing!

The real agenda is designer/programmer laziness. If you don't plan to be in the domain, you can say that it doesn't matter if you run as an admin, so you don't have to worry about permissioning, you don't have to engage with Windows management and the whole thing becomes very much more like DOS. Which is of course what all programmers want.

There are still reasons to leave a server out of your production domain.

• Because it is especially exposed to compromise. Servers in the DMZ might fall into this category. This is why we put application proxies of various sorts, in the DMZ, in front of domain member servers which are inside the firewall.
• In theory, because the hardening requirements are incompatible with domain management. But if so, why are you running Windows?
• Because, by design, this is not an area for your administrator team. The only case I've ever seen is retention of administrator access logs, where admin access would rather negate the point.

I reckon that's the lot.

The Emir

I shouldn't really comment on terrorism, or trials -- GNUWin32 is more my schtick. But I couldn't help noticing that all the reports of the second July bombers trial called Muktar Ibrahim the "emir" of the plot, and helpfully translated that word as "ringleader" or "kingpin".

I just wonder how the Foreign Office would take it if the BBC started writing about the Ringleader of Kuwait...

More GNUWin32

Just a quicky. If you need to dump a raw disk on windows, you can do it with the GNUWin32 dd(1) program. But

dd if=d:

won't work. You need a little sprinkle of stardust with the device reference:

dd if=\\.\D:

seems to do the trick.

Colours

Light grey sky with a gentle mottle. At my feet, I can see green June grass, but looking across the meadow to the ragged black-green hedge, the colour is lost under the floating carpet of pure yellow buttercups and smoky lilac ryegrass tassels.