Helping the World Through Software

Recently, I started talking with my girlfriend about the idea of writing a life plan.  The idea is similar in nature to a business plan, but instead of outlining the structure, mission statements, and strategies of a financial venture, you’re focused on the values, goals, and eventualities of your life as a whole.  I’ve researched a bit online, and the more I thought about it, the more I realized what a completely awesome tool a life plan could be – not only for organizing your life, but just the process of writing one can really illuminate and flesh out life-goals.  More importantly though, as I realized by talking with my friends, it can truly be a living document, one that grows over time as life, values, and situations change.

Though I am only in the planning stages now of what I want to include in my plan, I know before I put a single word down that there are two items that I will inevitably focus on.  The first is one of my true passions in life – creating.  Specifically, creating through computer science – games, AI, network utilities, or anything.  But ultimately I know this isn’t truly fulfilling.  I read article after tweet after blog post about software development and computer science – and some writing inspires me, and some falls flat.  It took me a while to figure out why, and as of late I realize more why that is.  Which brings me to the second item I will focus on – helping the world.  If I have a limited time on this big, blue globe, I want to do whatever I can to ensure that hopefully, at least in some small part, my creations will make the life a better place.  This – and making connections with other people who want to use their awesome skills to do some serious good!  I’m lucky enough to lots of friends with this attitude, and I’d love to make more.

Resources

To say there are a lot of amazing organizations out there changing the world on a daily basis would be an understatement – our lives change constantly with the evolution of social networks, mobile devices, and interconnectivity.  And while many of these changes attack very real problems and improve quality of life, there is still infinite amounts of space to effect positive change – still countless opportunities to do good.  And I think it’s important to deliberately focus on these items as a core goal.   I’ve recently begun to search online for resources and other like-minded buddies to help in this quest – and I’ve found a number in academia, as well awesome sites like TED that have some truly brilliant people focused on these very issues.

If you know of any other resources that talk about helping the world through computer science or other technology-driven philanthropy, please feel free to send them this way!

Or if you have any experience with writing a life plan or steps you’ve taken to clarify goals for yourself, please feel free to drop me a line!

I know there are other people much smarter than me who have tackled these areas before, so I’d love any guidance or tips.  I hope to continue to post on these subjects as I learn more and make further connections.

 

Finished up Multithreading in SynthNet

With BerryCraft done for now, I’m jumping back into SynthNet again – finished up multithreading today.  Now the emulator can utilize multicores/multiple processors, which allows substantially larger neural networks (which means more complex behavior and fun to be had).  There is still quite a lot of optimization to be done, but this was the biggie by far – I wasn’t really able to make anything much larger than the previous test until I got this working.

I changed the DNA from the demonstration a bit and grew a network with more neurons in the neural pathway, then had it listen to a piece of music (The Duel from Electric Dreams).  Captured a quick video of it – no practical demonstration, but fun to watch!

BerryCraft Source – Jave Minecraft I/O Engine

Below is the source to BerryCraft.  Most of the project is pretty trivial – however, MCIO.java (MineCraft IO) contains a class capable of full communication with a Minecraft server, correctly sending/receiving all Minecraft packet types as documented here.  BerryCraft itself only implements chat/time functionality (and logins of course), but the MCIO could be used to build out any Minecraft functionality (player positioning, mob spawning and attributes, inventory, etc).

If you do end up using MCIO, let me know, I’d love to see the results!

BerryCraft Source

 

 

BerryCraft Available for Download

I fixed up the last of the bugs, and added in a small amount of new functionality, and the first iteration of the BerryCraft, the Blackberry Minecraft chat client, is now complete.

 

The first iteration includes the following functionality:

  1. Connects to Minecraft server from your Blackberry (handles both authenticated or offline modes)
  2. Allows chatting with players online (and sending commands such as /time, /list, etc if available to your account)
  3. Has 3 user-assignable macros for repeatedly used messages/commands (good for admin commands)
  4. Allows user to set launcher and protocol version, which assuming no major protocol changes, should allow the client to continue to work with future server versions.
  5. Works with BB devices OS 5.0 and up over any connection type (Wifi, BIS, MDS, etc)

Additionally, the source code is available here – it includes an IO engine that fully sends/receives all packet types and a shell Game class to implement future functionality past chatting.

Download OTA here!

BerryCraft – Update 5/13/12

I was hoping to release the first iteration of the BerryCraft client tonight, and though I got a lot done this weekend, there are still a couple errors that are preventing it from going primetime.

Most importantly, I fleshed out 99% of the Minecraft protocol IO functions, so it can send and retrieve all 68 or so packet types as documented.  Obviously it only actually implements the basic connection/chat/time functionality (as opposed to drawing anything), but as far as communicating with the server, it understands everything (just  a couple errros still popping up from things not working right).  This is almost a necessity, as the server could potentially send any packet type over, and if the client doesn’t receive it properly, it will crash – so it has to be ready for anything in the protocol specification.  Internally, it can tell when mobs are spawned/move/look, when painting/items spawn, player abilities/movements, sound effects happen, etc etc.

Also – the client stays up for as long as desired (I ran it for 30-40 minutes at one point) – so it should be stable once the fixes are done.

Hopefully it won’t take more more than a couple nights to fix the bugs – expect another post when it’s good to go!

Authenticated Minecraft Logins Working for BerryCraft

A quick update – a lot of people had shown interest in BerryCraft – the Blackberry Minecraft admin/chat client I’d been working on.  It was a bit of a hack before and didn’t work with servers that required Minecraft.net authentication, so I wanted to fix it up before starting to release it out.   Tonight I successfully got the authenticated logins working.  I want to clean it up a bit more, and then stat releasing it in iterations.  The first will be mainly just a chat client, since that’s pretty much implemented.  Then we’ll see about inventory management and some other goodies if things work out well.

Quick Shoutout – ArtificialBrains.com

James Pearn at artificialbrains.com was nice enough to include SynthNet in his list of resources related to artificial intelligence.  Check out his site if you get a moment, it serves as a well laid-out directory of many neural network and other artificial intelligence projects going on around the world, as well as job listings.  Very cool site – thanks James!

 

Hardware Monitoring: Syncing Drupal with Zenoss

Overview

One of the more daunting tasks of managing a larger network is keeping track of all your devices – both physically, and from a network monitoring perspective.  When I arrived on the job 3 years ago, the first major task I laid down for myself was implementing both an asset management system, as well as a network monitoring system, to ensure we always knew what we had, and if it was functioning properly.

I decided almost immediately that Drupal was the right candidate for the job of asset management.  There are a number of commercial IT/helpdesk systems out there which work great, but they are usually fairly expensive with recurring licensing costs, and my history with them has always been shaky.  Plus, I find myself not always using all the functionality I paid for.  I knew with my Drupal experience, I could get something comparable up in almost no time – this is not a discredit to IT packages, but moreso the power of the Drupal framework.

Network Monitoring – Cue Zenoss

Now that I had the hardware DB taken care of, I needed a NMS for monitoring.  Originally I was planning on Nagios, but a contractor who works for us (now friend) had introduced me to Zenoss, another open source alternative.  Zenoss is awesome – is absolutely has its quirks, and is not the most intuitive system to learn, but once things are up and running it’s great – and tremendously powerful.  So the choice was made.

Now – I had both pieces, but I absolutely hate entering data twice, and the interoperability guy in me loves integrating systems.  So I decided to write a script that would sync our Drupal database with Zenoss.  Drupal would serve as our master system, and any hardware we entered into it would automatically port over to Zenoss.  Any changes or deletions we made (IP address, location, name, etc) would sync over as well.

The below script performs this synchronization.  Some warnings up front – I’m not a Python guy by any means, I specifically learned it for this script, so I apologize for any slopping coding or obvious Python-y mistakes.  I’ve tried to thoroughly comment it to document how to use it and how it works.  Hopefully it can help some others out as well!

# Description: Sync devices to be monitored from Drupal to Zenoss
#
# Setup Work: Create a (or use an existing) content type that houses your hardware items to be monitored.
# They should have CCK fields for the IP address of the device, the name, and the type of
# device it is. The device type will determine the Zenoss class the script adds it to, and hence
# the kind of monitoring it will receive (e.g. Linux server, switch, ping only, etc)
#
# Additionally, in Zenoss, create a custom property field that will house the nid of the Drupal
# node. This serves as the foreign key and will be used to link the item in Drupal to its entry in Zenoss
#
# Usage: This script should be run from zendmd, and may be run once or periodically. We run it every 20 minutes from
# a cron job.
# It will create new entries in Zenoss for items not yet imported, delete ones that no longer exist in
# Drupal (it will only delete ones that were originally imported from Drupal), and will update ones that have
# been updated (type, IP, location, etc).
#
# Note: Excuse all the extra commits - we experienced some issues with data not being saved, and I threw some extra in
# there - they're almost definitely not necessaryimport MySQLdb

# Take a taxonomy term from Drupal identifying the type of monitoring to be done,
# and convert it to the appropriate Zenoss class path. Update these to whatever terms
# and Zenoss class paths that make sense for your environment. We setup ones for
# Linux and Windows servers, switches, waps, UPSes, PDUes, etc, as can be seen.
def getClassPath(passType):

if passType.lower() == "windows":
return "/Server/Windows"
elif passType.lower() == "linux":
return "/Server/Linux"
elif passType.lower() == "switch":
return "/Network/Switch"
elif passType.lower() == "mwap":
return "/Network/WAP/Managed"
elif passType.lower() == "uwap":
return "/Network/WAP/Unmanaged"
elif passType.lower() == "ups":
return "/Power/UPS"
elif passType.lower() == "pdu":
return "/Power/PDU"
elif passType.lower() == "camera":
return "/Camera"
elif passType.lower() == "cphone":
return "/Network/Telephone/Crash"
elif passType.lower() == "sphone":
return "/Network/Telephone/Standard"
elif passType.lower() == "printer":
return "/Printer"
elif passType.lower() == "converter":
return "/Network/Converter"
elif passType.lower() == "ping":
return "/Ping"
return "/Ping"

# Connect to Drupal's MySQL DB (Replace these values with the appropriate ones for your system)
imsConn = MySQLdb.connect(DRUPAL_MYSQL_SERVER, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB)
imsCursor = imsConn.cursor()

# Execute the query to grab all your items to be monitored. In our case, we have a node type called "hardware" that had CCK fields identifying the IP address,
# the type of hardware (a taxonomy term that dictated the Zenoss class of the item - see getClassPath above), a physical location, etc.
# You'll want to change the specific table/field names, but the inner join will probably stay, as you'll want to grab both the node and CCK fields that belong to it.
imsCursor.execute("""
SELECT node.nid, content_type_hardware.field_hardware_dns_value, content_type_hardware.field_hardware_location_value, content_type_hardware.field_hardware_ip_value, content_type_hardware.field_hardware_monitor_type_value, content_type_hardware.field_hardware_switchname_value, content_type_hardware.field_hardware_switchport_value
FROM node
INNER JOIN content_type_hardware ON node.nid = content_type_hardware.nid
""")

# Loop through all returned records - Check for additions, changes, and removals
while (1):
#tempRow is our current hardware item record
tempRow = imsCursor.fetchone()
if tempRow == None:
# No more entries, break out of loop
break
else:
# Search Zenoss records for the nid of the hardware item. A custom field will need to be created in Zenoss to serve
# as this foreign key. In our case, we used MHTIMSID - but you can use anything you'd like - just be sure to create the field in Zenoss.
found = False
for d in dmd.Devices.getSubDevices():
if d.cMHTIMSID != "":
if int(d.cMHTIMSID) == tempRow[0]:
found = True
break

if found == False:
# Hardware item not found, add it if it's monitored
if tempRow[4] != None:
dmd.DeviceLoader.loadDevice(("%s.yourdomain.com" % tempRow[1]).lower(), getClassPath(tempRow[4]),
"", "", # tag="", serialNumber="",
"", "", "", # zSnmpCommunity="", zSnmpPort=161, zSnmpVer=None,
"", 1000, "%s (%s - %s)" % (tempRow[2], tempRow[5], tempRow[6]), # rackSlot=0, productionState=1000, comments="",
"", "", # hwManufacturer="", hwProductName="" (neither or both),
"", "", # osManufacturer="", osProductName="" (neither or both).
"", "", "", #locationPath="",groupPaths=[],systemPaths=[],
"localhost", # performanceMonitor="localhost',
"none")
tempDevice = find(("%s.yourdomain.com" % tempRow[1]).lower())
tempDevice.setManageIp(tempRow[3])
commit()
# Save nid to Zenoss record (to serve as foreign key) for syncing
tempDevice._setProperty("cMHTIMSID","MHTIMS ID","string")
tempDevice.cMHTIMSID = tempRow[0];
commit()
else:
# Hardware item found - delete, update, or do nothing
if tempRow[4] == None:
# Delete if not set to monitor
dmd.Devices.removeDevices(d.id)
else:
# Update DNS and IP to current values
if d.getDeviceName() != ("%s.yourdomain.com" % tempRow[1]).lower():
d.renameDevice(("%s.yourdomain.com" % tempRow[1]).lower())
if d.getManageIp() != tempRow[3]:
d.setManageIp(tempRow[3])
commit()

# Change class if not set to "Manual" (We setup a taxonomy term called "Manual" that would turn off automatic Zenoss class selection during syncing
# and allow us to manually specificy the class of the device.
if tempRow[4] != "Manual":
d.changeDeviceClass(getClassPath(tempRow[4]))
commit()

# Update comments (location change)
d.comments = "%s (%s - %s)" % (tempRow[2], tempRow[5], tempRow[6])
commit()

# Save any missed changes
commit()

# Close connection to database
imsCursor.close()
imsConn.close()

Watch SynthNet in Action!

In case you haven’t seen yet, the videos demonstrating SynthNet in action have been posted to YouTube!  In the first clip, I demonstrate growing a brain from virtual DNA, hooking into my Lego robotic buddy Bit, and then conditioning Bit to associate hearing a tone with getting his touch sensor pressed.  The demonstration is a recreation of classic fear conditioning experiments.

In the second clip, I give an explanation of how SynthNet functions, how the demonstration above was setup, and the future of the project.

Thanks for checking them out!

 

Site Makeover

After finishing up phase 1 of SynthNet, I came to the conclusion that I really missed updating the blog.  When I get involved in a project, I tend to get wrapped up in it (more accurately – completely and ridiculously obsessed where it takes over my life) and other things drop off the radar.  However, I’ve decided I want to make a real effort to not get AS wrapped up in projects, and remember to give the blog some love.

New and Improved!

As I went to write my first article after recording the SynthNet video, I also noticed the blog was looking a little tired.  They’d also made a number of improvements in WordPress since when I first installed everything, so I decided to take the leap, get a shiny new template, and put some new life into it.  I think it’s definitely an improvement – hope you enjoy it!