If you tell enough stories, perhaps the moral will show up.

Showing posts with label scripting. Show all posts
Showing posts with label scripting. Show all posts

2009-10-17

Logparser

Recently I wrote about the enumerate command that I use. I was looking at it just now because I wanted to enumerate one particular check across the whole domain: I wanted to report on the events that show a user being enrolled into local Administrators on their workstation -- and irregular admins generally.

This is a big deal for me -- has been for a long time, it's a big deal for more and more sites, and it should be for everyone. Admin privilege is the difference between spyware installing in a profile (and even now, most of them don't attempt to do this) and installing dangerously and ineradicably as a rootkit. Admin privilege is what allows  users to harm their builds with downloaded software or messing around with the branding or mapping. But alas, it's also the easy solution to a lot of problems and desktop team members -- admins themselves -- are often tempted to pass it on to a user in trouble so they can get on to the next call.

The control for this is to find out when it happens and follow up very promptly, next day, with the admin concerned. But you need to know it's happened, and the only ways I know how to tell it's happened are a) a listing of the group membership on every machine -- which doesn't, crucially, tell you when it was done, or b) the 536 message in the event log. So it's the message we want, provided we can pick and decode the content out of the rather unhelpful format. To hold the desktop team to account, we want to look at the new messages each day, making a nice report of all the suspect events.

We already have a tool -- enumerate -- which will run a command against every machine. So now we need a command that will append relevant log events on to a report. "But" I hear you cry, "but what about your RSA Envision log SELM appliance? Isn't that ideally suited to this task?" Well yes, my dears, it certainly is, but you see, it's licenced per event source. I have enough licences for all the infrastructure and about half of production servers, but none at all for workstations. We need something at a better price point, like free.

Microsoft is a better source of free (as in beer) software than you might expect, and they have the tool for this job: Logparser; motto: "the world is your database." In outline, Logparser converts and presents logs of many sorts and some odder stuff like registry and filesystem contents as queryable lists. The queries can be simple or complex: I started with

SELECT
 Strings 
FROM
 \\mypc\security 
WHERE
 EventID=536 
But you need to work a little harder to get a script parameterised enough to be enumerated across all domain members and produce a good outcome. The beauty of Logparser is that it's mature enough to deliver -- it really is a proper log analysis tool. I expected to write auxiliary scripts to break out the data, decode SIDs, accumulate the report as a CSV, and keep track of the last log read on each machine, but in fact all this can be done in Logparser script language or command line options.
-- admin.sql
-- Logparser query.
-- Accumulate events where a user has been made a member of admins or power users
-- You might want to enumerate this across the entire domain 
-- (omit domain controllers which have different messages)
-- Command would be like 
-- logparser 
--  -o:TSV -oSeparator:space -headers:OFF -fileMode:0 
--  -iCheckPoint:MYPC.lpc 
--  file:admin.sql?oFile=2009-10-18_AdminChanges+sMachine=MYPC
-- The checkpoint file is named for the machine, and output is appended to "today's" file.

SELECT 
-- Generating "hand" CSV rather than the CSV output type -- more flexible to do it in SELECT and USING
 
 -- the ms from the :ll aren't populated but it stops Excel dropping the seconds
 TO_STRING(TimeGenerated, '\"yyyy-MM-dd hh:mm:ss:ll\",')AS Date, 
 strcat(ComputerName,',') AS Computer,
 Resolve_SID (SID) AS Admin,
 Action,
 Resolve_SID (SIDUser) AS User,   
 Group

USING 
-- Do the token parsing in USING: break the bits we want out of the -|%{SID}|... tokens in Strings
 Extract_Token(Strings,1,'|') AS SUr,  -- User SID
 Extract_Token(Strings,2,'|') AS GroupN, -- (Localised for free -- more friendly)
 Extract_Token(Strings,3,'|') AS GroupD, 
 Extract_Token(Strings,4,'|') AS SGp,  -- the Group SID 
 
 SUBSTR(SUr,2,SUB(STRLEN(SUr), 3)) AS SIDUser,  -- break raw User SID out of the %{SID}
 
 CASE EventID WHEN 636 THEN 'enrolled' WHEN 637 THEN 'removed' END AS Action, -- Friendly EventIDs
 
 -- Output like "into BUILTIN\Administrators"
 STRCAT(
  STRCAT(
   CASE EventID WHEN 636 THEN 'into ' WHEN 637 THEN 'from ' END, 
   GroupD),
  STRCAT( '\\', GroupN)) AS Group
 
INTO
-- Need the -fileMode:0 (append) on the command line to avoid overwriting with each machine.
-- For a log for each machine then the command line above would let you use %Machine% in the name.
 %oFile%.csv 

FROM 
-- FROM the machine security log  --  This is -i:EVT. 
-- Don't use the SID resolve option because you may want to limit to particular built-in groups, but 
-- and S-1-5-32-544 is easier than working out internationalised versions of "Administrators"
 \\%sMachine%\Security 

WHERE 
 ((EventID=636) or (EventID=637)) and       -- 636 enroll, 637 remove
 (SID<>'S-1-5-18') and           -- Ignore actions by local System
 (                -- Ignore boring groups
  ((SGp = '%{S-1-5-32-544}') or (SGp = '%{S-1-5-32-547}')) -- Only want Admins or P Users
 -- Optionally don't report Domain admin (check your SID) being made admin, because it happens in every log!
 -- and 
 -- (SIDUser <> 'S-1-5-21-4163168572-49618088-4072775208-512') 
 )

Remaining niggles are petty. some machines have corrupt SELs -- logparser fails at end of log, so it never writes a checkpoint so the entire file is processed every time. But this can be fixed by saving and emptying the offending log. And I suppose it would be nice if it enumerated the domain itself, but that doesn't trouble me.

Apparently V3 is due out. I cannot wait.

2009-10-02

Fantasy Programming

I work at the command line, and I've never found a better place to do stuff in bulk. But since I've been dealing with dynamic networks of hundreds of PCs I've found that there's a tool missing from the utility set.

Windows is good for remote management -- better than people realise, but there's one thing it needs -- a decent network enumerator. What's that? It automates a task which crops up time and again -- running a command against a list of all the machines in a domain, or on a network, or subsets of those lists. I want to be able to type enumerate --domain MYDOMAIN.LOCAL --exclude "/(DC.*)|(PRNT.*)/" /cmd "mycommand %%Name%% %%Timestamp%% I think that's fairly clear: I want to enumerate the domain MYDOMAIN, exclude the DCs and those pesky HP print servers, and then, for each machine, run the command mycommand with the name of the machine and a timestamp on its command line.

I know that would be useful because I actually coded "enumerate" in Perl, and I use it a lot. But doing stuff in Perl has a limited future and I don't think there's really a pressing need to make it an editable script: the function seems rounded and complete -- not something that'll need continuous extension. So, as a first step to a .Net executable, here is my specification for the enumerate utility:


enumerate

 Enumeration settings
 All these settings can be combined and repeated to build up a list of hosts.
 Each entry is expanded into FQDN and IP and de-duped on both, with the last entry taking precedence.
 [--domain DN] Add all members of domain DN to the list.
 [--IP N/L|IP1-IP2] Add all ip addresses in the specified subnet
      (omit network and BC) or the specified range to the list
 [--list H[, H]*] Add all the H's (names or IP addresses) to the list
 [--flist "path"] Add all the hosts in the text file at "path" 
      (one per line, leading "\\" optional, blank lines allowed, anything after white space on a line is comment)
 Logging settings
 [--job name] Log all dignostics to the file called TS_run_name and all command output to the file TS_out_name.

 Command Settings
 [--cmd "string" [--[no]ping] [--[no]browse] [--omit "regexp"] [--directory "path"] [--concurrent nnn]]
 Run the specified string in a cmd shell, for each enumerated target.
 Multiple --cmds are allowed.

 --directory : cd to "path" before running -- default is "."
 --[no]ping : ping the host -- don't run if no response. Default is --ping
 --[no]browse : attempt to Windows browse the host -- don't run if no response. Default: --nostart
 --omit : don't run if the enumerated name or IP matches the given regexp (no //). Default: --omit ""
 --concurrent nnn : Run no more than nnn instances of this --cmd setting concurrently. Default: --concurrent 1

 "string" is the command to run. Default is "echo %%Host%% %%IP%% %%TS%%"

 Variables in the command string are expanded:
 %%Host%% -- The enumerated FQDN or IP address if it can't be resolved
 %%IP%%   -- IP address -- skip if a name can't be resolved
 Times -- all suitable for use in file names:
 %%Date%% -- The date the run started in ISO yyyy-mm-dd format
 %%Time%% -- The time the run started as hh-mm-ss
 %%TS%%   -- Now as yyyymmddhhmmsscc
Now that's a utility.

2009-09-12

Wireshark is OK, But a Bit Heavy

For sheer absence of dicking around, nothing beats tcpdump(1) which ought to be in your Linux install. I had to find out what systems were still using the old time server, which, happily was an ancient Slackware and, consequently, a Proper Operating System.

tcpdump -c 10000 port 123 > clockies

gathers the first 10,000 NTP packets, and, in another terminal session

gawk '/[0-9] 10/{print $2}' clockies | sort | uniq

gives you a list of the IPs (and you don't have to wait for the first one to finish.) Run wc(1) at the end of the second pipeline from time to time, while the first is still running, and you can see if any new IPs are cropping up.

What I like is the query language: for NTP packets there's just no more intuitive way of writing the search than "port 123"

SetACL for Command-Line Permissioning

SetACL looks like a saucy little alternative to approaches I've taken here before:
  • Unlike CACLS it uses a simple permission language -- no SDDL
  • Unlike CACLS and chmod(1) it works on services, registry keys, shares and printers
  • Unlike SubInACL it's not mental
I think I need to get familiar with this.....

2008-03-21

Last Logon Time

I have discovered a fascinating little gobbet of truth about the Active Directory 2003 records of the time users last logged on. The summary is this:

  • The Last Logon attribute tells us nothing useful. It's the time the user last logged on to the domain controller the query was run on. So if you run the query on a DC that doesn't do much authentication, you'll wonder why no-one has been logging on lately. Ignore this attribute unless you are gathering records from all DCs and selecting the latest...
  • The Last Logon Timstamp is different. It's the time the user last logged on to any DC in the domain. Aha! Problem solved? Sort of -- because the attribute is replicated across the domain, you'll get the same answer, give or take replication time, regardless of the DC you query. The only little fly in the ointment, leaching dark fluids and tainted chitinaceous fragments into the smooth white emulsion, is that replication time. It's not a minute. Or an hour or a day or a week. It's a fortnight.

So the proper interpretation of this field is something like this:

  • Blank: If the user has ever logged on, they must have done it in the last 14 days. Or perhaps they never have.
  • Date: The user definitely logged on on that date, and may have logged on any time up to 14 days after.
In practical terms, that means your script purging not-used-lately or totally-unused accounts can't delete an account on the basis of a blank Last Logon Timstamp. If you want to delete accounts that have never been logged on, you'll have to find another way.

2007-10-25

Read a Text File with Comments

This is a little Perl idiom to open a text file and read all the non-comment lines.

Key parts are

  • <LINELIST> yields all the lines in the filehandle into the default scalar $_
  • s/^\s*//; s/\#.*$//; s/\s*$//; removes (==substitutes nothing in place of) first any space at the front of the line second any trailing comment and third any trailing space in $_
  • chomp strips the newline from $_
  • next if /^$/ skips past the rest of the loop processing if $_ is empty (it matches beginning of line immediately followed by end of line)
All of this could be directed to a named scalar variable, but using the default pays off in compact code that can be cut and pasted elsewhere.
my $listfn="linelist.txt";
open LINELIST, "<$listfn" or die "Open $listfn $!";
while (<LINELIST>) {
    s/^\s*//; s/\#.*$//; s/\s*$//; chomp; next if /^\s*$/;
    print "$_\n"; # Or whatever else you want to do.....
}
Obviously printing is a bit dull -- instead you could drop the lines into an array:
push @linelist, $_;
Use grep to search it:
@results = grep((/$mypatt/i), @linelist);
The i after the pattern makes for case insensitivity, and you have to
use locale;
to get it right for all character sets.

2007-10-12

NMAP

This is a perl script that shows what perl does well. It takes a couple of programs which provide useful output -- nmap and nbtstat -- and combines them into a single daily archive showing what's on your network. Key features are:

  • The input is a list of named IP ranges.
  • The output is written to hosts.txt and yyyy-mm-dd.hosts.txt automatically building an archive of what's up in the networks you care about.
To run it
  1. Install nmap and make sure it's in the path. nbtstat is part of any Windows installation.
  2. Install Perl, which means Active State.
  3. Create a range file that's just a text file with a few lines like these: (Obviously with your network ranges. You can use any range specification that nmap will accept.)
      # my range.txt 
    servers    192.168.10.0/24 
    perimeter  192.168.12.0/24 
  4. Make sure your reverse DNS is working, or you won't get any DNS names.
  5. Run it with a command like c:\perl\bin\perl nmap.pl range.txt
  6. check the output in hosts.txt to see server names, and, for windows servers, domain and server names.
To get the benefit, set it to run every day using a scheduled task (I find this is easiest with a .CMD file containing full path names for everything) and in a few months you'll have some worthwhile history.

# Start of nmap.pl by umacf24
# This program takes a single parameter -- a file containing  pairs.
# Example (but leave off the #)
# servers    192.168.10.0/24
# perimeter 192.168.12.0/24
#
use strict;
use warnings;

# NBTSTAT codes from Jim Halfpenny
my %group  = (  hex("00"), "00 Dom",
  hex("01"), "01 M Browser",
  hex("1C"), "1C Domain Controller",
  hex("1E"), "1E S Browser", # Browser Elections
);
my %unique = (  hex("00"), "00 WS",    # Workstation service
  hex("01"), "01 Msgr", # Messenger Service
  hex("03"), "03 Msgr", # Messenger Service
  hex("06"), "06 RAS Server Service",
  hex("1B"), "1B Domain Master Browser",
  hex("1D"), "1D Master Browser",
  hex("1F"), "1F NetDDE Service",
  hex("20"), "20 SVR", # File Server Service
  hex("21"), "21 RAS Client Service",
  hex("22"), "22 MS EXC Interchange(MSMail Connector)",
  hex("23"), "23 MS EXC Store",
  hex("24"), "24 MS EXC Directory",
  hex("30"), "30 Modem Sharing Server Service",
  hex("31"), "31 Modem Sharing Client Service",
  hex("43"), "43 SMS Clients Remote Control",
  hex("44"), "44 SMS Administrators Remote Control Tool",
  hex("45"), "45 SMS Clients Remote Chat",
  hex("46"), "46 SMS Clients Remote Transfer",
  hex("4C"), "4C DEC Pathworks TCPIP service on Windows NT",
  hex("42"), "42 McAfee AV",
  hex("52"), "52 DEC Pathworks TCPIP service on Windows NT",
  hex("87"), "87 MS EXC MTA",
  hex("6A"), "6A MS EXC IMC",
  hex("BE"), "BE Netmon Agent",
  hex("BF"), "BF Netmon Application",
);

# results stored in an array for re-use
my @nmapout;

my $ofn='hosts.txt';
my $opath='.\\ip\\';

my $header = "Key\t++ Up but unnamed, \n\t** Named but not responding to ping.\n\t?? Improper NBT names\n";
$header .= "Ping scan run between ".localtime(time())." ";

my $ipall; my $ipup; my $ipdown; # Address counters

# Process the file of network names. Run NMAP against the network spec (iprange).
while (<>)
{
  next if (/^\s*#/) ;
  my @line = split /\s+/;
  if (2==@line)
  {
      my $ipname = sprintf "%-15.15s ", $line[0]; # Fixed width
      my $iprange = $line[1];      

      my $cmd = ".\\bin\\nmap -sP -R -oG - $iprange |";    # local nmap
      print STDERR "$cmd\n";
    
      open (NMAP, $cmd);
      while ()
      {
          chomp;
          next if /Smurf/;
          if (/^Host: /)
          {
              $ipall++;
              my $hostline=$_;
              $ipdown++ if ($hostline =~ /Status: Down$/);
              $ipup++ if ($hostline =~ /Status: Up$/);
              my $flag ='   ';
              $flag = '++ ' if ($hostline =~ /\(\)\tStatus: Up/) ;# Un-named
              $flag = '** ' if ($hostline =~ /\([^)]+\)\tStatus: Down/) ;# Named, but not responding
              my $hostip = $1 if ($hostline =~ /^Host: ([0-9\.]+) .+ Up$/);
              my $prefix=$ipname . $flag;
              my $postfix=nbtstat($hostip) if ($hostip);
              $hostline =~ s/^Host: /$prefix/;
              $hostline =~ s/Status: Up$/NBT: $postfix/ if ($postfix);
              # print STDERR "Host Line $hostline | $postfix | $hostip\n";
              push @nmapout, $hostline;
          }
      }
  }
}
$header .= "and ".localtime(time())."\n";

if ($ipall)
{    # Some addresses seen
  $header .= "$ipall addressses pinged:- Up: $ipup, Down: $ipdown\n";
}
else
{
  $header .= "No addresses scanned. NMAP in path?\n";
}

# Preserve an archive Version
my ($sec, $min, $hr, $day, $mon, $yr, $wday, $yday, $isdst) = localtime(time());
$yr = $yr + 1900;
$mon = $mon + 1;
my $prefix = sprintf "%04d-%02d-%02d.", $yr, $mon, $day ;
my $afn="$opath$prefix$ofn";
my $hfn="$opath$ofn";

#Don't overwrite existing hosts file until we have finished scanning.
open (AHOSTS, ">$afn") or die "could not open $afn $!";
open (HOSTS, ">$hfn") or die "could not open $hfn $!";
print AHOSTS $header; print HOSTS $header;
foreach my $hostline (@nmapout)
{
  print HOSTS "$hostline\n";print AHOSTS "$hostline\n";
}
close HOSTS; close AHOSTS;

sub nbtstat
{
  my ($hostip) = @_;
  my $cmd = "nbtstat -a $hostip |";
  my %survey; my $ret;
  my $domain; my $server;

  # print STDERR "survey $cmd\n";
  open (NBTSTAT, $cmd) or die "can't run $cmd $!";
  while ()
  {
      # print STDERR;
      # The meat of the NBTNAME command is in this format: name  type
      if (/^\s+(\S+)\s*<([0-9A-F]{2})>\s+(GROUP|UNIQUE)/i )
      {
          my $nbtname = $1;
          my $nbtclass = $2;
          my $nbttype = $3;
          if ('00' eq $nbtclass)
          {
              $domain = $nbtname if ('GROUP' eq $nbttype);
              $server = $nbtname if (('UNIQUE' eq $nbttype) && ($nbtname !~ /(\~)|(MSBROWSE)/));
          }
          my $desc = ('GROUP' eq $nbttype)? $group{hex($nbtclass)} : $unique{hex($nbtclass)};
          $desc = "$nbtclass $nbttype Unknown" unless $desc;
          # print STDERR "$nbtname $nbtclass $nbttype $desc \n";
          push (@{$survey{$nbtname}}, $desc);
      }
  }
  close NBTSTAT;
  # We should have a good domain name and a good server if we have anything
  if (%survey)
  {
      $ret = ($domain and $server) ?  "$domain\\$server" : '??';
  }
  foreach my $nbtname (keys %survey)
  {
      $ret .= " $nbtname: ("  . join (', ', @{$survey{$nbtname}} ). ")" unless ($nbtname =~ /(\~)|(MSBROWSE)/) ;
  }
  return $ret;
}
# End of nmap.pl

2007-09-27

Proxy Access for Services

Of course you want to use a web proxy, but some of your services need web access. Proxy settings are per-user, and if you run services as specific users you can log on and set them. But for the built-in anonymous accounts SYSTEM, SERVICE, how can you tell them where to find the proxies?

The obvious need for this is to get Windows Update working behind a proxy server. It's needed even if you are using the web interface, because WU still depends on the BITS service.

Well there are a number of ways. But what's easy is proxycfg, a command-line program that will create the appropriate entries in

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\WinHttpSettings

The program is in the XP build, but it runs fine on W2K -- just copy it over. Running it with the -u option will copy the current user's settings in to the service default and you're done.

Of course, you still need to ensure that the requests will be permitted by the proxy: The service can't authenticate. On our Bluecoats, you make a combined destination object that precedes authentication and is accepted on the first rule.

2007-06-15

More GNUWin32

Just a quicky. If you need to dump a raw disk on windows, you can do it with the GNUWin32 dd(1) program. But

dd if=d:
won't work. You need a little sprinkle of stardust with the device reference:
dd if=\\.\D:
seems to do the trick.

2007-06-01

In the Raw

Just as a glimpse of on-the-fly development to satisfy investigation needs, here's a hack using James Macfarlane's Windows registry parser to get a timeline of registry key timestamps.

This is a source code module -- no DLLs -- and so even though I've never been able to get ActiveState PPM to install CPAN modules, it's easy to set up. Just download, open the package and drag the components into the corresponding directory locations under C:\perl. Why not use TieRegistry or something? Because we need this to work on "dead" files and the Windows API won't do that. The extra benefit is that this will run on Linux.

Only remaining frustration: there doesn't seem to be a timestamp on values as well.

use strict;
use warnings;
use Parse::Win32Registry qw( :REG_ );
my $time_fmt = '%04d-%02d-%02d %02d:%02d:%02d';

my $usage="$0: hive_file_name\n";
my $fn=shift or die $usage;

my $registry = Parse::Win32Registry->new($fn);
my $root_key = $registry->get_root_key;

my %keytimes=(); 

sub keyinfo
{
    my $key = shift or die "no key to recurse";
    my $nm = shift or die "no name";
    my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst)=gmtime($key->get_timestamp);
    $year+=1900;$mon++;
    my $ts = sprintf($time_fmt,$year,$mon,$mday,$hour,$min,$sec);
    $keytimes{$ts." ".$nm}=[$nm,$ts];
    my @subkeys = $key->get_list_of_subkeys;
    foreach my $subkey (@subkeys) {
        keyinfo ($subkey, $nm."\\".$subkey->get_name);
    }
}
# Main execution starts here
keyinfo($root_key,'.');
foreach my $keytime (sort keys %keytimes) {
    print "$keytime\n";
}

Dumphive and the unicode registry strings

The handy dumphive utility will list out registry and SAM files, but a lot of the content is left as unicode strings represented as octet sequences like this:

"\\DosDevices\\E:"=hex:5c,00,3f,00,3f,00,5c,00,53,00,54,00,4f,00,52,00,41,00,\
  47,00,45,00,23,00,52,00,65,00,6d,00,6f,00,76,00,61,00,62,00,6c,00,65,00,4d,\
  00,65,00,64,00,69,00,61,00,23,00,37,00,26,00,31,00,66,00,65,00,39,00,65,00,\
  35,00,63,00,34,00,26,00,30,00,26,00,52,00,4d,00,23,00,7b,00,35,00,33,00,66,\
  00,35,00,36,00,33,00,30,00,64,00,2d,00,62,00,36,00,62,00,66,00,2d,00,31,00,\
  31,00,64,00,30,00,2d,00,39,00,34,00,66,00,32,00,2d,00,30,00,30,00,61,00,30,\
  00,63,00,39,00,31,00,65,00,66,00,62,00,38,00,62,00,7d,00
Well, you can pick your way through that with an ASCII table, but here's a bone-headed script to get the gist out.
use strict;
use warnings;
my $av=join(',' , @ARGV) ;
foreach my $c (split(/,+/,$av)){
    if (my $a=oct("0x$c")) {
        printf "%c", $a;
    }
}
It would be cooler to read the blocks directly -- backslashes and all. Maybe next time. Anyway, all you have to do is figure out what
\??\STORAGE#RemovableMedia#7&1fe9e5c4&0&RM#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}
means.

2007-05-27

GnuWin32 is the Shortcut for the Old and Feeble

I wrote about getting round the simple LUA bug in Steam. Essentially, the installer runs as an admin, which is fine, and it permits the entire installation directory to the admins only, which is not fine, because to use the app, you need write acces to at least some of the files. To get it working, and because the only documentation suggests that you have to be an admin to run steam, the simple route is to set permissions on that directory to Users/Full.

That's a bit of a challenge in XP on an isolated workstation, because the security tab is hidden, and on XP home, at least, it's hard to get it back. Last time, I bodged it with SubInACL. A bit like writing a program in COBOL to change the name of a file.

What I wanted to do was use chmod -- the Unix command. That's because I'm a crusty old fart, and it's hard for me to imagine anything easier than writing chmod -R a+rw "C:/program files/steam/" That may look involved, but the Windows command line equivalent, cacls, is way tougher and you have to use a different program to make it recurse. That's why I ended up with SubInACL.

I'm not the only person to prefer to work this way, but be stuck using Windows. (Because I am, OK?) A lot of people turn to CygWin -- a complete *nix environment hosted on Windows. That's good -- you get shells, utilities, compilers, familiar filing system, the lot, but the very compatibility makes it alien within Windows, and it's really too much of a commitment for me.

So instead, I move further back along the compatibility spectrum, and get to GnuWin32, and that makes me happy. Essentially I can download the stuff I want: grep, less, the core command line tools like ls and chmod, and OpenSSH without making a big production out of pretending to be on Unix. The working ports come as nice friendly installers and the only manual step is to set your path. Because it's not Cygwin, there's no issue using these tools alongside Activestate Perl, or Windows scripting. And the license is impeccable, of course.

Perfect? Almost. Some problems just can't be mentioned ("where's the GNU vi? Tee hee!") and some systems would suit me but aren't there. (I know RCS is obselete, but I don't want to learn SVN.) But GnuWin32 is part of the toolkit.

2007-04-10

Bedtime

As it's the holidays, the kiddies would stay on the PCs all night. To get them into bed, stern measures are needed:

First you need to set the accounts they are using (you're not letting them be admins, are you?) to have fixed logon hours. You can't do this through the GUI on XP Home, so you need a batch file of commands like this one to set times when it's possible to log on. Call it accounts.cmd -- you'll need to re-run it when anything changes:

net user mmadson /times:sunday,08:00:-21:00;monday-saturday,09:00-21:00;saturday,08:00-21:00
The /passwordreq:no directive can be useful here too.

Unfortunately, Windows won't enforce that logoff. (A domain would, but Windows itself will not.) So the second step is to force it. There would be any number of ways to deal with this, but I chose the ugliest: run the the Sysinternals psshutdown command at bedtime. I chose to run it from a command file so that I could get a log. Put this text into enforcer.cmd, make the obvious modifications and set it up as a Windows scheduled task. (In the control panel, under Performance and Maintenance.)

@echo off
echo "-start-" >>d:\at\log.txt
date /t >>d:\at\log.txt
time /t >>d:\at\log.txt
d:\at\psshutdown -o -f  >>d:\at\log.txt 2>&1
echo "-end-" >>d:\at\log.txt
I set it to run twenty minutes past the last log on time. Hey presto! Instant rage from the younger generation.

The obvious improvement is to only log off console sessions which are members of the kiddiewinks group. It's really annoying when it logs ME off! I think I could script that up, but I'm too idle.

2007-02-11

Light Feet on the Drive

OK -- the scenario is that you really, really want to know what's on a Windows workstation hard drive -- you plan to look in the IE cache, system logs, registry, SAM, etc. You can't/won't be arsed to image it and work on the image and you are not going to take this rather urgent moment to learn about excellent Linux based tools. But you do want to take all reasonable precautions. (What's reasonable? I'm less sure than I was after reading this document. It's a normally reliable source, but the example scenario contains an eyepopping amount of work on a live system. Maybe evidence rules are different in the States. My approach is to kill the disk and only ever read from it.) Here's the plan.

  1. Prepare an investigation machine. You need a computer with Internet access where you can work privately. You also need a USB disk housing that will fit the disk in question. Maplin do an IDE/SATA for 3 1/2 inch disks, while 2 1/2 inch laptop disks still seem to be small format IDE and there are lots of housings for those. Since we really don't want to write to the evidence disk, run the Read Only registry file below, and test that you can't write to a scratch USB device. Load tweakui (Microsoft Powertoys) and make sure that you're not set to autoplay anywhere to reduce the risk of malwaring your investigation machine.
  2. Give the job a name. The Remedy number, "2007 02 Hotmail Complaint" -- whatever.
  3. Get a chain of custody log. The idea here is that you have a collection of evidence for the investigation, and as you collect each item, you sign it out and and back in when you return it. so that you can swear to where anything was at any future tribunal.
  4. Get a log book. Or open a file, or something, anything where you can write everything down. Computer records are good here as you can paste in log entries and images. Finish each day with next steps so you don't forget, then print the day's record, and sign and date each page. Enter it into your evidence store.
  5. Pull the power on the workstation. Record make model and serial number. Remove the disk, and record the make, model and serial. Put this diskless carcass into your evidence store with a label that says "2007 02 Hotmail Complaint Exhibit A". You shouldn't need to boot it, but you never know. Anyway it's evidence.
  6. The disk is Exhibit B. Log it, and sign it out to yourself. Mount it in the USB housing. Check that you've run readonly.reg on your investigation machine. Plug it in and make sure it comes up on you're investigation machine. Don't let it auto play.
  7. Where you go now is up to you. Check the tools below to look at Windows file contents, and there are others to look at file times.
Read Only.reg

Create a file called readonly.reg using notepad. Save it on your desktop. The file contains just these lines:


Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\StorageDevicePolicies]
"WriteProtect"=dword:00000001
Double click on the file and confirm you do want to load the settings. Then test on a scratch USB stick -- you should see a "Write Protect" warning come up when you try and save something. To get back to read/write you need another file with that last dword set to 00000000.

Read Event Log Files

The log files you want will be in d:\windows\system32\config\ assuming d: is where your disk is. Sometimes you can load the .evt files from the subject disk into the log viewer. I find that it always says they're corrupt though.

So I use Activestate Perl, add the Parse::EventLog module and a few lines of code to list them out into an easy text format. Here's the code -- you'll want to tweak it.

use strict;
use Parse::EventLog;
$|=1;

my $elogfn = 'd:\\windows\\system32\\config\\SecEvent.Evt';
print "Loading Event log: $elogfn .." ;
my $elog = Parse::EventLog->new($elogfn);
print "..loaded\n";
my %c = $elog->getOldestEvent();
while (%c = $elog->getNextEvent())
{
  my $str;
  if ($c{Strings})
  {
      $str = join('|', @{$c{Strings}}) ;
      $str =~ s/\t/\\t/g;
      $str =~ s/  / /g;
  }
  my $evt = $c{EventID};
  my $time = localtime($c{TimeGenerated});

  my $ msg = "$time: $evt <$str>\n";
  print $msg unless grep({$_ == $evt} (560, 576, 515, 600));
}

The good bit here, is that this will work from a Linux machine just as well as Windows.

Read Registry and SAM files

The most amazing thing thing I've learnt recently is that the SAM is in the same format as a registry hive. This means you can use this tool to print out the system registry and the SAM (from d:\windows\system32\config\) as well as user registry from ntuser.dat in the appropriate profile.

You should also be able to use Parse::Win32Registry though I haven't done that. It would work from Linux, too. There's scope for a useful script here, as the SAM is in a desperately unhelpful format.

2007-01-23

The .Net Developer's Guide to Windows Security

Just a quick note to big up Keith Roberts and his book "The .Net Developer's Guide to Windows Security". Don't be foxed by the ".Net" bit -- it just means "Modern Windows". The whole text is online but it's well worth buying the book simply because it's broken down into five and ten minute chunks, so that a couple of month's toilet visits will have you knowing more than you want about every major topic in Windows Security.

It's not a management or policy text and knowing all of it is not obligatory, but this book has helped me deal with issues that ended up on me because apparently I know this stuff. I don't -- I stole it from Keith. Just as examples, check out How to Develop Code (and do other stuff) as a non-Admin or How to Store Secrets on a Machine

Serious, practical stuff.

2007-01-15

Reporting Permissions

I've been spending the quiet period after Christmas working on some scripts. For some time now I've felt the need for a report of permissions, hierarchically by directories in the DFS, and also per trustee. Reports like that would let me see a lot of things I want to see:

  • Permissions on users: no users should have direct permissions
  • Permissions on uncontrolled groups: if it's not a ROLE group it shouldn't deliver any access in the departmental filing
  • Everyone/all users permissions: almost never correct
  • I could run them on servers as part of the compliance checks
They would also remind me that there are things I want to address:
  • ACEs referring to deleted accounts and groups -- There's a handy subinacl option to remove this
  • "Domain Admin" permissions -- one of the techs here solved admin file access problems that way -- when what he needed was local administrator permissions
  • Permissions delivered by SID-historied groups that need to be dropped or replaced
A good thing all round. The only problem is that they are very hard to get. It seems like there would be tools to do this, but I can't find anything suitable. Only after spending days hunched over a hot interpreter have I found why: it's moderately difficult, and the semantics of the report are surprisingly tricky.
  • The simple approach doesn't work. If you build a hierarchy and read and report the ACL masks for each file and directory, the output is unusably difficult to understand. Even on one object there can be multiple ACEs for each trustee.
  • And there's just too much data. Even if people could understand the access options implied by mask bits and types, there are ten million objects on our file servers. No-one reads a ten-million-line report.
The approach I've come up with with pretty crude, but I'm hoping it's going to do something useful:
  1. Don't try and represent the subtleties of permissions. Boil everything down to (none) Read, Write and Deny in that increasing order of priority. For any given trustee, only even think of reporting the highest priority. Fancy stuff like "deny execute, allow read" just shows up as the single "highest" permission: in this case it's Deny. (Oh yes, at this level, Deny is a permission, regardless of DACL type flags and ACE masks)
  2. Don't distinguish between inherited access and directly granted. What matters for reporting is the actual access. When you come to rectify, you'll need to know how it got there, but the Windows tools are good for that.
  3. Do use the concept of inheritance to trim down the report. The only permissions you need to report are where something changes. If a trustee gets a permission in the root of a ten-thousand-directory, 300-thousand-file volume, and every file and directory inherits it or has had it applied, then your report for that trustee is one line, not a third of a million lines.
  4. Assemble your basic reports for each trustee -- it makes the purging much easier. If you want to report a single directory structure with entries for all trustees, you can mash that together later.
  5. I've found three levels of interest in files (when they need a report at all)
    • Leave them off entirely -- seems brutal but it cuts the run-times and you don't lose much
    • Report them as a single aggregated pseudo-name "[one or more files]" for each directory -- this effectively raises a warning if any file is more permissive than the settings on its directory
    • Report each individually following the same rules as for directories
  6. Expect to translate between the names you use to extract the permissions, and the names you report. Consider a DFS for example.
The down side of all this simplification is this: it's simple. That Everyone/Full permission that appears at the root and inherits all the way down is highly significant, but easy to miss. After all, it'll only be the one line. I think this means that we still need to apply automatic policy exception detection, but that is a project for the future.

2006-07-01

Why Perl?

It's looks like line noise, and if it was ever in fashion it's dropped out now. But Perl suits me, and I think this is why:

  • Some people see the world as tables or XML -- I see it as text files with easy to parse lines
  • Security does a lot of work with "fairly regular" data. (It doesn't seem possible to get the admins to stick to strict group naming conventions.) Putting regular expressions at the heart of the language acknowledges that the data are a bit dodgy.
  • Security has many command line utilities that do roughly what you want. Perl runs external code, gathers output, skips the irrelevant bits and tidies up the good lines, all without too much pain.
  • I've never written a right-first-time program in any other language. (I don't think I've ever written a right-at-all program in any language where I have to do my own garbage collection.)
  • Languages that let me say what I want get my vote:
    $a++ unless ($its_time);
    foreach ( <STDIN> ) {reformat($_)};
Perhaps I'll grow out of it. Perhaps I'll just get frustrated with weak Windows integration. Perhaps I'll write that integration the way it should be done. Perhaps the Active State port will blow up once too often. We'll see.

2006-05-23

Time for SubInACL

Do we script because we are old enough to remember when the command line was all there was? Or are we so old that we don't feel that there's time to muck about any more?

Either way, I've been trying to find a way to make bulk changes in file server permissions. These are typically volumes of a few hundred GB with something like one to ten million objects. I need to apply Chinese Walls (a term of art, not an architectural reference), apply them now, and the helpdesk is only halfway through the permissioning process implementation that would have let me do this properly.

Well, it's time to use the dreaded Deny permission. Easy to say, but tougher to apply to millions of objects past unpredictable inheritance, Creator Owner permissions and distinctly dodgy admin permissions. I've tried a good many approaches:

  • It's obviously got to be a script.
  • To convince the auditors, the permissions have to go on to the filesystem roots of "all" servers, and adjust the denied groups on the way down. A file of UNCs and allowed business units is being prepared as I write.
  • I don't like that hourglass up for hour after hour. I like it even less, knowing that the changes I'm making will be silently abandoned every time it encounters a break in inheritance
  • I already have alldisks.pl to enumerate the UNC of every disk on every (matching) server from a domain or a list, and run a command against it....
  • And ultimately, I'll want to take it off, once we have the permissioning process up and running properly and honestly

I can't find a perl module that lets me do this. Win32 Security looks good, but I'm too stupid to make it work -- it boggles without builtin admin/Full. Filesystem Object is not really my area, but it seems to completely lack DACLs

The obvious tool is is [X]CACLS, except that I can't make it go past inheritance breaks, so the script has to chase it down the tree, testing each layer to see if the applied ACE has got there. And that's no joke when the output is SDDL.

SubInACL is about editing ACLs, not adding new ACEs. Isn't it? Oh.

Yes. SubInACL has grown up. The latest version (and believe me, you really need the latest version -- the one in the 2K3 resource kit doesn't even work) provides a robust, tree-oriented structure to report, grant or deny permissions at 100,000 objects per hour on any remote or local server where you are a local admin. Sure, the command language is a bit bonkers, the report output needs serious digestion to be useful for people, and the management of the ACL inherit flag preserves that same maddening ambiguity. But that's why we have Perl, and I can live with it all, just for the sake of knowing that my changes will be applied the way I write them. The fact that I can fix some stupid global admin access control, and do it for free in the same pass as my deny permissioning is just a huge bonus.

Almost for certain, SubInACL will do what you want, and if it won't, I'll bet that what you want isn't legitimate. If you couple it with Win32::NetAdmin for remote management of local groups, you can be in a better place for scripted permissioning than you would ever have believed.