Planet SoC -

GSoC is over… Kinda

rhinovirus — Tue, 19 Aug 2008 03:41:10 +0200

Wow, Google Summer of Code is kind of over and I’ve had a great summer with Nmap. When I say its kind of over I mean that I still have to submit my end of summer quiz, still ahve to submit my code to Google, and I still have to wait for my mentor to submit his quiz. And there is still a chance - though infinitesimally small - that Google opts not to pay me or send me a T-Shirt for my work.
I have good news and bad news, the good news is that I finished all the projects I wanted to before the deadline, the bad news is I wont get my T-Shirt until after I go back to school. Which means I can’t use it to gloat over everyone the first day back; damn, I usually reserve that day for being an asshole.
Looks like I wont be making a good first impressions this year
In other news, you may notice a new link in the sidebar. I decided to take advantage of GoDaddy’s free SSL for Open Source offer. Hopefully - soon - you will be able to brows my site securely.
Finally, I plan on releasing more of the code that I have written recently, hopefully someone can use my scraps.

GSoC is over… Kinda

rhinovirus — Tue, 19 Aug 2008 03:41:10 +0200

Wow, Google Summer of Code is kind of over and I’ve had a great summer with Nmap. When I say its kind of over I mean that I still have to submit my end of summer quiz, still have to submit my code to Google, and I still have to wait for my mentor to submit his quiz. And there is still a chance - though infinitesimally small - that Google opts not to pay me or send me a T-Shirt for my work.
I have good news and bad news, the good news is that I finished all the projects I wanted to before the deadline, the bad news is I wont get my T-Shirt until after I go back to school. Which means I can’t use it to gloat over everyone the first day back; damn, I usually reserve that day for being an asshole.
Looks like I wont be making a good first impressions this year
In other news, you may notice a new link in the sidebar. I decided to take advantage of GoDaddy’s free SSL for Open Source offer. Hopefully - soon - you will be able to brows my site securely.
Finally, I plan on releasing more of the code that I have written recently, hopefully someone can use my scraps.

GSoC is over… Kinda

rhinovirus — Tue, 19 Aug 2008 03:41:10 +0200

GSoC

rhinovirus — Fri, 15 Aug 2008 00:34:17 +0200

The Summer of Code is almost over and I still haven’t written any(enough) documentation!!!
Ill have to rectify this immediately.

Nmap Codeswarm

rhinovirus — Mon, 30 Jun 2008 21:05:51 +0200

Jabra was nice enough to make a Codeswarm out of the Nmap codebase.
The result is a tad skewed because it doesn’t include the /nmap-dev, /nmap-exp, or /ncat repositories where some big new features are being coded but its more or less accurate and fun to watch.
Check it out here.

New Nmap diff format

rhinovirus — Sun, 15 Jun 2008 22:19:01 +0200

For my readers that don’t know what Nmap is, you wont understand the following. I’ve been gun preparations to start coding an Nmap diff utility (codename Ndiff), and one of my tasks was to propose a possible format for the diff files, enjoy the following.
Hey everyone,
David said it better then I can:
On Sun, Jun 15, 2008 at 3:02 PM, David Fifield <david@bamsoftware.com> wrote:
> The real problems is that Zenmap’s comparison doesn’t answer the
> questions a users wants answered: Are there new hosts on the network?
> Did any machines go down? Any new ports? Web server still running?
> Zenmap just gives you a jumble of colored text and asks you to figure it
> out.
Nmap could use a program that intelligently compare XML output files,
instead of just doing the type of diff that Zenmap currently uses, we
could be parsing the files and outputting an intelligent diff that
better reflects the differences in network state. This diff file could
then be used by Zenmap or a third party program for visualization.
Here is a partial example of how the differences could be represented
in XML. The new tags <addhost> <delhost> <chghost> are used to express
changes in the host state. Inside the *host tags <chg(state/service)
is paired with <old(state/service) to show what has changed, both <add
and <del (port/state/service) are unpaired because their changes are
obvious.
<nmapdiff startdate="12/4/07" starttime="13:31:42" enddate="12/4/08" endtime="21:04:26" >
<chghost>
   <address addr="10.9.8.7" addrtype="ipv4" />
   <addport protocol="tcp" portid="53">
      <addstate state="open" />
      <addservice name="domain" product="ISC BIND" version="9.2.1" method="probed" conf="10" />
   </addport>
   <chgport protocol="tcp" portid="80"> <state state="open" />
      <chgservice name="http" product="Apache httpd" version="2.0.39" conf="10" method="probed" />
      <oldservice name="http" product="Microsoft ISS" version="11" conf="10" method="probed" />
   </chgport>
</chghost>
<addhost>
   <address addr="192.168.2.1" addrtype="ipv4" />
   <addport protocol="tcp" portid="21"> <addstate state="open" />
      <addservice name="ftp" product="ftpd" version="1" method="probed" conf="10" />
   </addport>
   <addport protocol="tcp" portid="8080"> <addstate state="open" />
      <addservice name="http" product="Apache httpd" version="2.0.39" conf="10" method="probed" />
   </addport>
</addhost>
<delhost>
   <address addr="192.168.2.100" addrtype="ipv4" />;
   <delport protocol="tcp" portid="21"> <delstate state="open" />
      <delservice name="ftp" product="ftpd" version="1" method="probed" conf="10" />
   </delport>
   <delport protocol="tcp" portid="8080"> <delstate state="open" />
      <delservice name="http" product="Apache httpd" version="2.0.39" conf="10" method="probed" />
   </delport>;
</delhost>
<runstats>
   <addhost address="192.168.2.1" reason="reset"/>
   <delhost address="192.168.2.100"/>
</runstats>
</nmapdiff>

The differences could also be outputted in a more readable format such as:
12/4/07 - 13:31:42 -> 12/4/08 - 21:04:26
10.9.8.7:
   Port 53/tcp is now open, was filtered
   Port 53/tcp is now listening with ISC BIND
   Port 80/tcp is now listening with Apache httpd, was Microsoft ISS
192.168.2.1:
   Host 192.168.2.1 has come online.
   Port 21/tcp is now listening with ftpd
   Port 8080/tcp is now listening with Apche httpd

There is also the matter of the language I would implement this in,
Ive given it some thought and heres what I’m thinking:
PERL
Pros:
- Many parsers and applications for Nmap’s XML format already written
in this language.
Cons:
- Nmap would have to list PERL as an optional requirement.
Python:
Pros:
- Easy integration with Zenmap
- Nmap already relies on Python for Zenmap, no extra dependencies.
Cons:
- It seems that the only Nmap XML parser(xml-expert) is dead.
- I’m not particularly fond of python.
C++:
Pros:
- Easy integration with Nmap/Zenmap
- Possible future integration for XML parsing with Nmap, resuming
scans and etc.
Cons:
- I couldn’t find any prior work, so I would probably end up working
with LibXML2
As you can see, there are still a lot of unresolved details, I would
appreciate any input on this. Here are some of the requirements that
David gave me:
* It must read Nmap XML files. (It is explicitly not a requirement to
read any other type of output file.)
* It must compare two output files at a time.
* It must show which hosts have come up or gone down.
* It must show when a port has changed state.
* It must support comparing output files coming from different scans
(different Nmap options or different target specifications).
* It must allow output in English text.
* It must allow output in XML, with a format and DTD to be determined.
Cheers,
Michael

New Nmap diff format

rhinovirus — Sun, 15 Jun 2008 22:19:01 +0200

<nmapdiff startdate="12/4/07" starttime="13:31:42" enddate="12/4/08" endtime="21:04:26" >
<chghost>
<address addr="10.9.8.7" addrtype="ipv4" />
<addport protocol="tcp" portid="53">
<addstate state="open" />
<addservice name="domain" product="ISC BIND" version="9.2.1" method="probed" conf="10" />
</addport>
<chgport protocol="tcp" portid="80"> <state state="open" />
<chgservice name="http" product="Apache httpd" version="2.0.39" conf="10" method="probed" />
<oldservice name="http" product="Microsoft ISS" version="11" conf="10" method="probed" />
</chgport>
</chghost>
<addhost>
<address addr="192.168.2.1" addrtype="ipv4" />
<addport protocol="tcp" portid="21"> <addstate state="open" />
<addservice name="ftp" product="ftpd" version="1" method="probed" conf="10" />
</addport>
<addport protocol="tcp" portid="8080"> <addstate state="open" />
<addservice name="http" product="Apache httpd" version="2.0.39" conf="10" method="probed" />
</addport>
</addhost>
<delhost>
<address addr="192.168.2.100" addrtype="ipv4" />;
<delport protocol="tcp" portid="21"> <delstate state="open" />
<delservice name="ftp" product="ftpd" version="1" method="probed" conf="10" />
</delport>
<delport protocol="tcp" portid="8080"> <delstate state="open" />
<delservice name="http" product="Apache httpd" version="2.0.39" conf="10" method="probed" />
</delport>;
</delhost>
<runstats>
<addhost address="192.168.2.1" reason="reset"/>
<delhost address="192.168.2.100"/>
</runstats>
</nmapdiff>

The differences could also be outputted in a more readable format such as:

12/4/07 - 13:31:42 -> 12/4/08 - 21:04:26
10.9.8.7:
Port 53/tcp is now open, was filtered
Port 53/tcp is now listening with ISC BIND
Port 80/tcp is now listening with Apache httpd, was Microsoft ISS
192.168.2.1:
Host 192.168.2.1 has come online.
Port 21/tcp is now listening with ftpd
Port 8080/tcp is now listening with Apche httpd

There is also the matter of the language I would implement this in,
Ive given it some thought and heres what I’m thinking:
PERL
Pros:
- Many parsers and applications for Nmap’s XML format already written
in this language.
Cons:
- Nmap would have to list PERL as an optional requirement.
Python:
Pros:
- Easy integration with Zenmap
- Nmap already relies on Python for Zenmap, no extra dependencies.
Cons:
- It seems that the only Nmap XML parser(xml-expert) is dead.
- I’m not particularly fond of python.
C++:
Pros:
- Easy integration with Nmap/Zenmap
- Possible future integration for XML parsing with Nmap, resuming
scans and etc.
Cons:
- I couldn’t find any prior work, so I would probably end up working
with LibXML2
As you can see, there are still a lot of unresolved details, I would
appreciate any input on this. Here are some of the requirements that
David gave me:
* It must read Nmap XML files. (It is explicitly not a requirement to
read any other type of output file.)
* It must compare two output files at a time.
* It must show which hosts have come up or gone down.
* It must show when a port has changed state.
* It must support comparing output files coming from different scans
(different Nmap options or different target specifications).
* It must allow output in English text.
* It must allow output in XML, with a format and DTD to be determined.
Cheers,
Michael

First nmap merge. keep it simple.

rhinovirus — Fri, 30 May 2008 06:07:02 +0200

I applied my first major Nmap patch today, Yay! major milestone. From my patch email:
I have just applied a patch that reorganizes where the ping
probe information for -PS, -PU, -PA, and -PO is stored. Previously
this information was stored haphazardly in the global class ‘NmapOps’,
I moved them into the ’scan_lists’ struct which isn’t stored as a
global.
Ideally this wont change any functionality so if something is acting
weird with respect to the probes listed above, it may be my fault.
The most likely places a bug may appear would be in traceroute.(cc/h),
idle_scan.(cc/h), service_scan.(cc/h), services.(cc/h), targets.cc,
and scan_engine.cc. I have gone through all these functions thoroughly
but it is possible that I missed something.
Cheers,
Michael
About 80% of the settings in Nmap are stored internally as global variables, it helps out a lot in simplifying how Nmap works (anything to help simplify a program that complex is welcome) but in some cases it can also block new features. For instance, it is imposable to specify a unique range of scan ports for each host in your scan; when you scan multiple hosts they all must have the same - globally stored - parameters.
Its kind of disappointing due to how powerful Nmaps parallelism is; if you need to specify two different option sets for two different host scans you effectively have to start Nmap twice. While this might be trivial for two hosts, imagine 10,000 hosts, each one with unique scan options; you cant possibly start each nmap scan at the same time due to memory limitations so the only options are to ether take a lowest common denominator type scan that encompass all options, or brute force the list a few commands at a time.
But to rectify this would require a massive redesign of how Nmap fundamentally functions - with the chance of introducing a massive amount of bugs into this pretty stable software package - and would necessarily complicate the command line interface to a point that might be overwhelming to a new user(lets not forget about GNU/Hurd).

Fundamental software design problem:

Make sure you keep the software in a state that all users can use it
Add awesome new feature that can do your taxes and clean the kitchen sink
Keep the software in a state that no normal user can use the advanced features without adding any new functionality and start charging $1,500 per seat *COUGH*Nessus*COUGH*

Cheers,
Michael

First nmap merge. keep it simple.

rhinovirus — Fri, 30 May 2008 06:07:02 +0200

Fundamental software design problem:

Make sure you keep the software in a state that all users can use it
Add awesome new feature that can do your taxes and clean the kitchen sink
Keep the software in a state that no normal user can use the advanced features without adding any new functionality and start charging $1,500 per seat *COUGH*Nessus*COUGH*

Cheers,
Michael

Google sent me a free book

rhinovirus — Fri, 16 May 2008 01:22:39 +0200

Free as in free beer.