Category Archives: Tools and Techniques

Posts in this category are about tools and techniques that I find useful during a Penetration Test

Esedbxtract

TLDR: Python script to automate the extraction of hashes from ntds.dit and system files. Available here : https://bitbucket.org/grimhacker/esedbxtract.

During an internal Penetration Test, once I’ve gained Domain Administrator access the fun doesn’t stop. In order to test the strength of user account passwords I need to retrieve the password hashes.

There are several ways to do this with either specialist tools or builtin Windows utilities as  @lanjelot discusses here and Inquis discusses here and here.

My preferred method is to use Volume Shadow Copy to extract a copy of the NTDS.dit and SYSTEM files, since this is an administrative task carried out with Windows utilities it does not normal cause alerts or require the Antivirus to be disabled – as is the case with some of the other options.

Once I have these files on my Kali box I use esedbexport from libesedb to export the data and link tables, these tables are used with the SYSTEM file by ntdshashes.py (available here by LaNMaSteR53 based on ntdsusers.py from ntdsxtract) to get the hashes.

This method can take a long time if the Active Directory is very large, but at this point during the assessment I’m not usually in a rush, it is less intensive on the domain controller, and it doesn’t panic the IT staff of the organisation with alerts.

There are a few reasons why I wrote a script to automate the extraction of the hashes from the NTDS.dit and SYSTEM files.

First and foremost, it’s a multi step process that doesn’t actually require any brain power – just for the output of one tool to be fed into another in the correct way. Writing a script to do this means that time isn’t wasted waiting for me to come back to it between each step. And of course it is another success for my continuing mission to replace myself with a small script…

Second ntdshashes.py didn’t work with the the latest version of ntdsxtract when I last rebuilt my machine (because a ntdsxtract added a new required working directory parameter). I patched the script (and raised an Issue on the repository) but decided it wouldn’t take much work to rewrite it as a class and include it in a larger tool.

Finally I hadn’t played with calling external programs as a subprocess in python before and this seemed like a reasonable excuse.

The result is esedbxtraxt.py available on BitBucket: https://bitbucket.org/grimhacker/esedbxtract.

Dependencies are stated in the README.

Normal usage will result in the password hashes being written to ‘hashes.pwdump’:

python esedbxtract.py -n /path/to/NTDS.dit -s /path/to/SYSTEM

Use ‘–help’ or see the README for further options.


As always, if you have any questions, comments or suggestions please feel free to get in touch.

 

Bitfields

I recently had need to interpret bitfields with Python.
I’m quite happy with the 3 lines of code that I came up with so I thought I’d share them in case they are of use to anyone else.

Bitfields are basically a binary number where each bit is assigned a meaning which can either have a value of True ‘1’ or False ‘0’.
Usually they are interpreted using bit shifting and bitwise AND operations but this seemed to be quite involved to get the data into a usable form so I found another way.

Consider the pwdProperties attribute from Active Directory (http://msdn.microsoft.com/en-us/library/ms679431(v=vs.85).aspx) which contains several settings for the account as a bitfield which can be retrieved using an LDAP query.

Each of the bits of this attribute mean the following:
1st bit = DOMAIN_PASSWORD_COMPLEX
2nd bit = DOMAIN_PASSWORD_NO_ANON_CHANGE
3rd bit = DOMAIN_PASSWORD_NO_CLEAR_CHANGE
4th bit = DOMAIN_LOCKOUT_ADMINS
5th bit = DOMAIN_PASSWORD_STORE_CLEARTEXT
6th bit = DOMAIN_REFUSE_PASSWORD_CHANGE

So if the pwdProperties attribute has a value of 17 in decimal, which equals 010001 in binary, the 1st and 5th bits (from the right) are set to 1 indicating that the domain requires complex passwords and stores passwords in cleartext.

Using python-ldap this attribute is returned in a dictionary as a decimal number represented as a string within a list, i.e.

attrs = {'pwdProperties': ['17']}

So the first step is to extract the string of the number and convert it to an integer:

pwd_properties = int(attrs['pwdProperties'][0])

Next the decimal number is converted to a string representation of the binary number with left 0 padding to the correct length:

pwd_properties = format(pwd_properties, "06b")

Then the binary number string is split into a list:

pwd_properties = list(pwd_properties)

For my purposes I needed the bitfields to be represented as a boolean. To do this a the string replace() method is used to replace instances of ‘0’ with an empty string and then the bool() function is used to convert the result to either True or False while iterating over the list. (Note when dealing with strings an empty string is False and everything else is True).

bitfield_values = [bool(w.replace('0', '')) for w in pwd_properties]

Next a list containing the meaning of each bit is defined (make sure you have them in the correct order to match the bits) :

bitfield_keys = ['refuse_password_change', 'password_store_cleartext', 'lockout_admins', 'password_no_clear_change', 'password_no_anon_change', 'password_complex']

The two lists can then be formed into a list of tuples using zip() which is then used to create a dictionary using dict() :

pwd_properties = dict(zip(bitfield_keys, bitfield_values))

Finally this can all be condensed into :

bitfield_keys = ['refuse_password_change', 'password_store_cleartext', 'lockout_admins', 'password_no_clear_change', 'password_no_anon_change', 'password_complex']
bitfield_values = [bool(w.replace('0', '')) for w in list(format(int(attrs['pwdProperties'][0]), '06b'))]
pwd_properties = dict(zip(bitfield_keys, bitfield_values))

Resulting in a dictionary like this:

{'password_store_cleartext': True,
'password_no_anon_change': False,
'lockout_admins': False,
'refuse_password_change': False,
'password_no_clear_change': False,
'password_complex': True}

A limitation of this method is that it is not easy to go from the resulting dictionary back to the bitfield because a dictionary in Python is unordered. This can probably be overcome by using an ordered dictionary from the collections module. However for my current purpose there is no advantage to implementing this.

EDIT:
I have been mulling this over and come up with the following line to convert the dictionary back to a binary number :

int("{refuse_password_change}{password_store_cleartext}{lockout_admins}{password_no_clear_change}{password_no_anon_change}{password_complex}".format(**pwd_properties).replace('True', '1').replace('False', '0'), 2)

This is probably horribly inefficient due to the string replacement, but it works.
It takes advantage of unpacking and referencing keyword arguments to form a string with the values in the correct order, then replaces the strings ‘True’ and ‘False’ with ‘1’ and ‘0’ respectively before using the int() function to convert the string base 2 number (i.e. binary) to a decimal number.

It might be more efficient to avoid string replacement like this :

int("{refuse_password_change}{password_store_cleartext}{lockout_admins}{password_no_clear_change}{password_no_anon_change}{password_complex}".format(**dict(zip(pwd_properties.keys(), ['1' if pwd_properties[key] == True else '0' for key in pwd_properties.keys()]))), 2)

This recreates the dictionary with ‘1’ and ‘0’ by testing each key for True. Then takes advantage of unpacking and keyword arguments to get the bits in the correct order, before converting to a decimal number using int().

At some point I’ll time these two methods to find which is more efficient and update this post.


As always, if you have any comments or suggestions please feel free to get in touch.

Nettynum – A Windows Domain Enumeration Tool

TL;DR: I’ve written a Windows Domain Enumeration Tool, you can get it from here: https://bitbucket.org/grimhacker/nettynum.


Enumerating information from the Windows Domain is nothing new and has been quite extensively covered in books, blog posts and academic papers.

So what is the point off this post you ask? To release my first tool to the community!

There are a lot of very good tools to extract information from the Windows Domain, some of which have been especially created for use by Penetration Testers while others were created for administration.

However when on tests there are a lot of things that I need to be doing and taking the output of one tool, and feeding into another is not one of them. I wanted a tool that I could set running with no information and have it find everything it could about the Windows Domain, including the Domain Name, Domain Controllers, Domain Groups, Domain Admins, and Accounts Policies.  i.e.

“Replace myself with a small shell script.”

This lead me to create Nettynum, a Python script that automates enumeration of information from the Windows Domain for my final year project at Northumbria University. It currently uses the pywin32 to access the Windows API functions and is therefore a Windows only tool – but I have plans to address this.

For those interested all 190 pages of my dissertation including UML diagrams can be found here: ‘Oliver Morton Individual Project Nettynum‘.
It was accompanied by the first version of Nettynum (those au fait with git can checkout the first commit in the main branch of the BitBucket repository).

The automated domain enumeration option will:

  1. Discover the NetBIOS and DNS style domain names on the network.
  2. Discover the domain controllers for these domains (IP address, NetBIOS name, and DNS name).
  3. Authenticate to the domain controller (with a NULL session by default).
  4. Enumerate a List of Domain Groups (and their comment).
  5. Enumerate the members of groups that match a regular expression (.*admin.*) by default.
  6. Also attempt to enumerate the members of certain groups (“Domain Admins” and “Enterprise Admins” by default) regardless of whether they were in the list of groups or not. – This is to work around the 100 group limit of the Windows API.
  7. For each of the discovered users, enumerate the account information (including comment, whether it is disabled, SID,  Bad Password Count, Password Age, and more.)
  8. Enumerate the groups of which these users are a member.  (At the time of writing I am not aware of another tool that does this.)
  9. Enumerate the Accounts Policies (Lockout Threshold, Duration and Observation Window, Minimum/Maximum Password Age, Minimum Password Length, and Password History Length).
  10. Deauthenticate from the domain controller.
  11. Output this information as an XML file (so it can be easily parsed for use by other tools [I have ideas, watch this space]) with a style sheet applied (so that the file can be opened in a browser and read easily by a human).

So in short, if the domain controller has NULL sessions enabled this:

python nettynum.py -A

Results in this:

There is also an automated option for ‘local’ enumeration which discovers hosts and then proceeds to enumerate information on a ‘local’ level before generating a report.

Nettynum also offers targeting options to limit the scope to specified Domain Names, Domain Controllers, or Groups. It also allows the user to specify authentication credentials, or will use a previously established session with the host (and leave it intact afterwards).

Finally there is also a ‘manual’ mode which allows the user to specify a particular category of information to enumerate but requires prerequisite information to be provided.

Where to get it: clone or download from BitBucket https://bitbucket.org/grimhacker/nettynum

Dependencies and usage are in the README file.

I have no doubt that there are bugs hiding in the code partly due the nightmare that is Windows documentation and partly because of my mediocre programming skills, so if you find any please let me know!


As always, if you have any comments or suggestions please feel free to get in touch.