Perl Automation Saves the Day - 6 of them in fact!

How does one analyze 182,000+ lines of Excel looking for correlations? I suppose there may be some Excel magic, but when IP addresses are involved - and I know Excel hasn't had an out-of-the-box IP network sort order, methinks I'm in for a long slog.

The background: I have an export from a customer IP address management tool and we needed to verify the block type assignments were correct. To do this, I needed to understand the top-level aggregates (IP network and mask) for each top-level container (site). There were around 50 aggregates in each of 6 containers. I then needed to verify if the 30,000+ subnetworks in each container did or didn't fall within the 50 associated aggregate blocks.

Where to begin? Looking at, analyzing and deciding on 1 line per second would still take almost 6 and a half working days (8 hours / per day); and 1 per second is a gross underestimate of the actual time required. I needed to automate this somehow.

Notepad++, DBGP and ... Python!

My recent work with Notepad++ got Perl debugging working with the DBGP Plugin. Originally written for PHP debugging, I did a little tweaking on the Perl side and got it working.

I don't know a lick of Python, but could I get Python to work with the DBGP Plugin too? Worth a shot!

++ for Notepad++

I've been doing some work lately with Notepad++ to get Perl debugging working. I used NppExec to automate a debug command from the Macro menu. But why stop there?

I dove headfirst into customizing Notepad++ into a full fledged Integrated Development Environment (IDE) for coding. I mainly do Perl, but also some C work and lots of Windows batch files. I added an NppExec script to "Compile", "Compile and Run" and "Clean" in build directories. It points to a pretty complex and branching Windows batch file that determines the correct action based on provided arguments and the current file extension.

How does this all work?

Debugging Perl Debugger: Part 3 - Automation

We have Perl debugging integrated with Notepad++. We fixed variable values so they show up in watch lists. Now we want to automate the use of Perl debugging directly from Notepad++ while editing a Perl script.

NppExec is a plugin with powerful automation features for Notepad++. The documentation showed it had all I needed to get this to work.

Debugging Perl Debugger: Part 2 - Variable Values Vindicated

In yesterday's post, I described how I got Perl debugging integrated with Notepad++. I had issues with watch variables only showing the variable name and type, not the value. I had source code from both the DBGp plugin and the Komodo Perl debugger. My only real option was to change the Perl debugger source as the plugin was written in a language I'm unfamiliar with and didn't have a compiler for.

The DBGp plugin offers a cool button labeled "DBG" which pops up a window with the raw XML messages sent between the debugger and the plugin. Like a packet capture trace file, this was a great place to start. Adding a watch variable and querying it's state showed the XML exchange and specifically where the value was returned:

...
&ltproperty name="$VERSION" fullname="$VERSION" encoding="base64" type="scalar" constant="0" children="0" size="25" &gt
    &ltvalue encoding="base64"&gt
        &lt![CDATA[MS4wIC0gMjkgSlVMIDIwMTU=]]&gt
    &lt/value&gt
&lt/property&gt
...

So it was getting sent. Why couldn't the plugin read and display it?

Debugging Perl Debugger: Part 1 - Notepad++ Integration

I use Notepad++ for most of my Windows text-based editing including Perl scripts. Every once in a while I'm stuck and need debugging, but I was too lazy to learn the Perl debugger command line. I had a simple Perl-specific IDE laying around solely to use for the integrated debugging in its Graphical User Interface (GUI). The ideal solution would be Perl debugger integration with Notepad++.

Google searches revealed there was no Perl-specific debugger integration in Notepad++; however, there was a debugger plugin - DBGP Plugin listed on the Notepad++ Plugins list. It was originally written for PHP and Xdebug integration. Further searches and the README lead me to believe it may support other programming languages as long as they supported the DBGP - common debug protocol. Could Perl work this way?