I'll be frank. This post is not going to be pretty, elegant or overall very fun to read. This is not your fault, readers. I am not punishing you. In fact, this particular entry is being written as a response to a software development project that I was recently and currently involved in. Unfortunately, until 7 hours ago, the project was, more or less, not fully implemented and thus there was a bit of difficulty in gathering my final thoughts on the ultimate experience I wanted to convey.
To be brief, the project's inconsistency most likely stemmed from a difficulty in facilitating an effective line of communication between our team members, likely due to the fact that we are all current college students with too much on our plates and too little time in the day. However, this ultimately resulted in quite a bit of "last minute heroics". I have to admit, I am largely to blame. At the inception of the project, I was full of optimism and may have unwittingly assumed a role of leadership I was not necessarily willing to be in. As a result, as my initial barrage of Emails missed their marks, I let my own failure to "rile up the troops", so to speak, get to me. As a result, my own enthusiasm for the project diminished significantly and now I sit before you in confession of having given up even before the "last minute heroics".
Overall, my experience of issue driven project management was hectic. In theory, it sounds like an amazingly streamlined and intuitive way of solving the project management problem. However, in practice, if project members are unable to communicate effectively, the entire process completely deteriorates. This is, of course, not the problem of the design of the concept but rather a problem with the people trying to take advantage of the process. At times, it seemed like a big hassle to create an issue just to work on something but, again, this problem arises only due to a lack of consistent group meetings.
The command-line interface our group has implemented is fully functional and includes four commands for the users to interact with the WattDepot data energy at the Hale Aloha towers on the UH campus. Our design allows for the project's main execution to only create a processor, which then essentially takes over. In retrospect, having the main method exist outside of the processor is probably unnecessary, but in the spirit of keeping things modular, this was the design choice we made. The processor then explicitly "gathers" commands in a HashMap. Although I say "gathers" here, our implementation actually requires that we hard-code the existence of these commands into the processor. However, barring adding a couple of lines to the class, the system is modular and adaptable to accept additional commands.
Although I feel like the overall quality of the project is pretty good and is something I can reflect on and take some pride in, I am certain that, in the past 7 hours (6 of which I was asleep for), there was a significant amount of testing done to the completed project as a whole. Additionally, I found it really troublesome to write JUnit test cases for many of the classes I implemented, due to functionality conflicts like void methods or the infinite loop which drives the command-line interface. I was able to get fairly decent test coverage on the classes by making them more modular and testable but I am still very reluctant to say that the tests were "good". This perception does not, however, factor in ease of development for new developers, ease of use for users or even if the project can be successfully distributed because, again, despite having waited to the last minute, I ran out of time to fully test these requirements.
Tuesday, November 29, 2011
Tuesday, November 8, 2011
Introduction to WhatDepot?
In my last post, I intended to bring to everyone's attention the energy situation in Hawaii and how, in the midst of a unique problem, there are unique solutions. I also touched upon the fact that there exist numerous opportunities for a software developer to contribute as part of these solutions, especially in terms of data analyses.
As a concept, preparing a system for energy data analysis seems like a trivial task. Install a power meter, read that meter at some interval, store that data into a database and then query the database. However, when the time comes for implementation, it becomes a very involving task indeed. Luckily, we live in a time where open source applications are numerous and it's often easy to find an open source project that suits your needs, even if only partially so. Shortly after my previous entry, I learned of an open source web service which collects electricity data and stores it in a database. The service, aptly named WattDepot, even features a robust API for accessing the collected data. WattDepot can even be installed locally for experimentation or simulations and is capable of near real-time feedback. Of course, the best part of WattDepot might be that it's open source, and thus free to use for anyone who is interested in taking advantage of its many features.
But, as with almost all open source projects, there is certainly a bit of a learning curve involved. When I first tried my hand at creating a Robocode robot, there was quite a bit of trial and error as well as a heavy dose of perusing the Robocode API. WattDepot was no different. Although I had access to a WattDepot server which was already set up, and I knew that I had a wealth of predefined tools to help me interact with it, learning exactly how to do what I needed to do still took a bit of the proverbial banging of my head against the wall. Once again, in order to ease into a new system, I performed several katas to help familiarize myself.
1. Get sources. Print alphabetically.
One of the most basically things I learned quickly about WattDepot was that all the data came from sources. Each source had a variety of information associated with it, and one of the most useful bits of information is its name. In trying to alphabetize these sources, I tried to take advantage of the fact that the source names are Strings, and the String class's natural ordering happened to be, in a sense, alphabetical. Here, I "cheated" a bit and threw all the sources into a treeSet to shed the duties of sorting. Then the realization came that even if I were to sort the names, I wouldn't necessarily be able to keep each source's description associated with it. Luckily, Source objects in WattDepot have a compareTo method built in, and as fate would have it, the comparison was by name.
2. Get sources. Print from newest to oldest data.
The second thing I quickly learned about WattDepot was that there were SensorData objects associated with the sources. In order to get the timestamps of how recent the latest data was obtained from a particular source, I had to obtain the latest SensorData associated with each source. Here, I learned again how the compareTo method for this data type was implemented by throwing all the SensorData objects into a treeSet and printing them out. As luck would have it yet again, the objects were sorted by how "fresh" the data was, and printing out the name of the source the data came from was a simple exercise in parsing the String representation of the source.
3. Get sources. Print out their subsources. And their subsources. And their subsources...
With a heavy dose of String manipulation review in my mind now thanks to all the data parsing from the previous exercises, it wasn't particularly difficult to get split a source's subsources up and print them out. However, there was a bit of trouble here because in order to print out hierarchies, it made sense to make use of recursion. Unfortunately, upon investigation, all of the sources on this particular server had either only one level deep of subsources or none at all. What this means is that, even though I wrote my hierarchy printing code to be prepared for recursion, I was never able to fully test it out, since there is no recursion involved if there were no subsources that had its own subsources.
4. Get sources. Print energy consumed by that source... yesterday.
This particular kata or simple exercise took, by far, the longest of all. The goal was very similar to the first two exercises, yet the implementation had to be drastically different. When a query to the WattDepot database for the energy consumed by a particular source is made, the result is just the energy as a floating decimal number. In order to sort these numbers, I chose to put them all into an ArrayList. However, keep the numbers relevant, I needed a second ArrayList with the names to which the energy consumption data corresponded. Only then was I able to sort the lists and print out the results. This kata, however, took a longer time to implement because I hadn't had a lot of experience manipulating Calendar objects, but it turned out to be a surprisingly powerful and useful tool.
5. Get sources. Print highest recorded power per source... yesterday.
With my new found understanding of Calendars, and having already set the foundations for filling two lists which correspond to one another in order to sort them, this kata took a bit shorter to implement. However, due to the fact that we needed to find the highest recorded power by each source, we had to sequentially query the database a number of times per source in order to compare and find the highest recorded number for each source. This not only added more loops, decreasing the performance of the application, but added a lot more individual queries to the server, and overall, the execution of this particular kata was almost excruciatingly slow. In order to expedite the process, I decreased the query intervals to a mere two queries per source and when I was satisfied that the code was serviceable, I reverted back to shorter intervals between queries. Unfortunately, much like the conundrum between exhaustive and practical testing, I was only able run practical tests with longer intervals so as to avoid waiting for long periods of time just to test the execution.
6. Get sources (I see a pattern here). Print the average energy used in the last two Mondays.
Given the last two katas, the most trouble this one gave me was actually in the date manipulation. I found that, if you tried to set the Calendar object's day of the week to Monday, you will end up getting the upcoming Monday's date. What I needed was the past Monday's date. After wracking my brain briefly, I ended up going with the brute force method of the good ol' switch statement. If today is Monday, go back seven days. Otherwise, if today is Tuesday, go back one day and so forth until Sunday, where we go back six days. No, this was not the most elegant solution, but given how little I still understand about how Calendar objects fully function, it was a workable solution.
Through these exercises, I got to understand WattDepot a lot more and, as much as I hate to admit it, I learned a lot about Java that I may not have otherwise learned. While it is on a crude level so far, a lot of the energy data manipulation I learned from these exercises involved sorting data and formatting the output. This is deceptively important, however, because a big part of studying data involves presenting the data in ways that the data can be visualized and trends and patterns might be identified. I was able to complete these six katas to my satisfaction but I unfortunately neglected to jot down the specific amounts of times each one took to implement because I tend to do my coding with breaks interjected. Ultimately, however, it isn't the amount of time devoted to learning a craft that matters, but what you are able to derive out of the experience instead. In this case, I learned a lot more about WattDepot and how to manipulate the information I can attain from interacting with it by sitting down and devoting time to learning and I truly hope that I can inspire you to do the same. Even if it's not WattDepot, or even computer science... sit down and take the time to learn something today! After all, if I hadn't put in the time and effort, I'd definitely still be asking "what depot?"
As a concept, preparing a system for energy data analysis seems like a trivial task. Install a power meter, read that meter at some interval, store that data into a database and then query the database. However, when the time comes for implementation, it becomes a very involving task indeed. Luckily, we live in a time where open source applications are numerous and it's often easy to find an open source project that suits your needs, even if only partially so. Shortly after my previous entry, I learned of an open source web service which collects electricity data and stores it in a database. The service, aptly named WattDepot, even features a robust API for accessing the collected data. WattDepot can even be installed locally for experimentation or simulations and is capable of near real-time feedback. Of course, the best part of WattDepot might be that it's open source, and thus free to use for anyone who is interested in taking advantage of its many features.
But, as with almost all open source projects, there is certainly a bit of a learning curve involved. When I first tried my hand at creating a Robocode robot, there was quite a bit of trial and error as well as a heavy dose of perusing the Robocode API. WattDepot was no different. Although I had access to a WattDepot server which was already set up, and I knew that I had a wealth of predefined tools to help me interact with it, learning exactly how to do what I needed to do still took a bit of the proverbial banging of my head against the wall. Once again, in order to ease into a new system, I performed several katas to help familiarize myself.
1. Get sources. Print alphabetically.
One of the most basically things I learned quickly about WattDepot was that all the data came from sources. Each source had a variety of information associated with it, and one of the most useful bits of information is its name. In trying to alphabetize these sources, I tried to take advantage of the fact that the source names are Strings, and the String class's natural ordering happened to be, in a sense, alphabetical. Here, I "cheated" a bit and threw all the sources into a treeSet to shed the duties of sorting. Then the realization came that even if I were to sort the names, I wouldn't necessarily be able to keep each source's description associated with it. Luckily, Source objects in WattDepot have a compareTo method built in, and as fate would have it, the comparison was by name.
2. Get sources. Print from newest to oldest data.
The second thing I quickly learned about WattDepot was that there were SensorData objects associated with the sources. In order to get the timestamps of how recent the latest data was obtained from a particular source, I had to obtain the latest SensorData associated with each source. Here, I learned again how the compareTo method for this data type was implemented by throwing all the SensorData objects into a treeSet and printing them out. As luck would have it yet again, the objects were sorted by how "fresh" the data was, and printing out the name of the source the data came from was a simple exercise in parsing the String representation of the source.
3. Get sources. Print out their subsources. And their subsources. And their subsources...
With a heavy dose of String manipulation review in my mind now thanks to all the data parsing from the previous exercises, it wasn't particularly difficult to get split a source's subsources up and print them out. However, there was a bit of trouble here because in order to print out hierarchies, it made sense to make use of recursion. Unfortunately, upon investigation, all of the sources on this particular server had either only one level deep of subsources or none at all. What this means is that, even though I wrote my hierarchy printing code to be prepared for recursion, I was never able to fully test it out, since there is no recursion involved if there were no subsources that had its own subsources.
4. Get sources. Print energy consumed by that source... yesterday.
This particular kata or simple exercise took, by far, the longest of all. The goal was very similar to the first two exercises, yet the implementation had to be drastically different. When a query to the WattDepot database for the energy consumed by a particular source is made, the result is just the energy as a floating decimal number. In order to sort these numbers, I chose to put them all into an ArrayList. However, keep the numbers relevant, I needed a second ArrayList with the names to which the energy consumption data corresponded. Only then was I able to sort the lists and print out the results. This kata, however, took a longer time to implement because I hadn't had a lot of experience manipulating Calendar objects, but it turned out to be a surprisingly powerful and useful tool.
5. Get sources. Print highest recorded power per source... yesterday.
With my new found understanding of Calendars, and having already set the foundations for filling two lists which correspond to one another in order to sort them, this kata took a bit shorter to implement. However, due to the fact that we needed to find the highest recorded power by each source, we had to sequentially query the database a number of times per source in order to compare and find the highest recorded number for each source. This not only added more loops, decreasing the performance of the application, but added a lot more individual queries to the server, and overall, the execution of this particular kata was almost excruciatingly slow. In order to expedite the process, I decreased the query intervals to a mere two queries per source and when I was satisfied that the code was serviceable, I reverted back to shorter intervals between queries. Unfortunately, much like the conundrum between exhaustive and practical testing, I was only able run practical tests with longer intervals so as to avoid waiting for long periods of time just to test the execution.
6. Get sources (I see a pattern here). Print the average energy used in the last two Mondays.
Given the last two katas, the most trouble this one gave me was actually in the date manipulation. I found that, if you tried to set the Calendar object's day of the week to Monday, you will end up getting the upcoming Monday's date. What I needed was the past Monday's date. After wracking my brain briefly, I ended up going with the brute force method of the good ol' switch statement. If today is Monday, go back seven days. Otherwise, if today is Tuesday, go back one day and so forth until Sunday, where we go back six days. No, this was not the most elegant solution, but given how little I still understand about how Calendar objects fully function, it was a workable solution.
Through these exercises, I got to understand WattDepot a lot more and, as much as I hate to admit it, I learned a lot about Java that I may not have otherwise learned. While it is on a crude level so far, a lot of the energy data manipulation I learned from these exercises involved sorting data and formatting the output. This is deceptively important, however, because a big part of studying data involves presenting the data in ways that the data can be visualized and trends and patterns might be identified. I was able to complete these six katas to my satisfaction but I unfortunately neglected to jot down the specific amounts of times each one took to implement because I tend to do my coding with breaks interjected. Ultimately, however, it isn't the amount of time devoted to learning a craft that matters, but what you are able to derive out of the experience instead. In this case, I learned a lot more about WattDepot and how to manipulate the information I can attain from interacting with it by sitting down and devoting time to learning and I truly hope that I can inspire you to do the same. Even if it's not WattDepot, or even computer science... sit down and take the time to learn something today! After all, if I hadn't put in the time and effort, I'd definitely still be asking "what depot?"
Tuesday, November 1, 2011
Sim City: Hawaii Edition
If you've ever played Sim City, you know how hard it can be to be to make those decisions as mayor to help your tiny town grow into a bustling metropolis. I'm not talking the decisions about which disasters to unleash so you can watch your virtual citizens be engulfed in its destructive glory (try the alien invasion, aliens always tear your city up good). Nay, I am talking about the legitimate decisions of balancing your city's need for utilities, the cost required to maintain them, as well as environmental concerns.
One of the utilities that your Sim City citizens craved the most (when they weren't getting blasted by alien laser beams) is, of course, electricity. Although the decision in the game was one of simply keeping citizens happy by providing them with electricity while not contaminating their breathing air with pollution, the struggle for keeping a city out of the dark is a much more complex situation in reality. There are other concerns to look out for as well, such as the depletion of non-renewable resources that threatens to spell the end of the era of taking the availability of fuel for our cars or oil to burn in generating electricity of granted.
At this point, you might be wondering what this post is doing on my software engineering blog. The answer is two-fold. First, since you are reading this blog at all, you are on an electronic device with access to the internet and the concept of power and energy should at least interest you on a superficial level even if it's not a lingering thought in the back of your mind all the time. Secondly, this is my blog and I can write whatever I see fit on here!
Having said all that, allow me to express something of importance to this entry. I live in Hawaii. This is important because Hawaii, unlike most places in the country, faces a very unique situation when it comes to energy. Due to its relatively isolated positioning in the world, Hawaii is heavily dependent on importing oil in order to keep our lights on. What this amounts to is that our residential electricity prices lead the nation by nearly triple. It's clear to see now how the energy challenges that we face in Hawaii might concern a software engineer whose livelihood depends on having consistent access to electricity. Even putting aside the obvious reliance of a software engineer upon power, it is also important to take note of Hawaii's unique challenges from a concerned citizen's perspective.
In addressing Hawaii's energy situation, it is good to first understand that energy is the measure of power used over a given amount time. In other words, a light bulb using a little power that's on for a long time can end up using the same, or even more, energy as a microwave that uses a lot of power but is only on for a few minutes in the day. Energy consumption is therefore what we must keep our eye on even more so than how must power a device requires to operate.
Energy conservation is a concern all over the world, but it is of particular importance in Hawaii. As I mentioned previously, Hawaii's reliance on oil for energy generation raises the costs of energy quite a bit. Additionally, due to the fact that Hawaii is geographically isolated from the rest of the world, its energy grid is isolated as well. This means that energy can't be freely transferred from grid to grid as needed as is the case in the continental U.S., not to mention the fact that Hawaii's power plants need to be smaller and therefore less efficient as well. What all of this amounts to is essentially that traditional methods of power generation aren't nearly as effective in Hawaii as they are elsewhere, yet it is very difficult to uproot these practices and adopt new methods as well.
Granted, it is often difficult to buck traditions and change is often slow and sometimes painful. Fortunately, there is and should be optimism for the energy situation in Hawaii because, along with its unique energy problems, Hawaii also has unique potential solutions. For example, although Hawaii's small size results in inefficient power grids, the same small size means a more modest need for energy overall compared to larger metropolitan areas. Furthermore, Hawaii is uniquely primed for numerous sources of renewable energy as well, including wind, wave, solar, ocean thermal and geothermal. Couple the potential for renewable energy with the desirability for Hawaii and its people to ditch the reliance on imported oil and there certainly exists reason for optimism.
Of course, better people than I have thought of these things before me and it is through these people that I am able to convey some of these thoughts to you today. It would especially be of surprise to me if any of these concepts were lost on the folks at the Hawaii Clean Energy Initiative (HCEI). The HCEI ambitiously plans for Hawaii to achieve 70% clean energy by the year 2030. There is probably initially quite a bit of skepticism for this number upon hearing it. However, the plans call for 40% of the energy to be from renewable sources and also reducing our need for energy by 30% thus enabling the 40% of renewable energy to effectively be 70% of the overall energy requirements. As people grow ever more conscious of the growing need for an energy solution and become more efficient with their energy use, even by simply replacing light bulbs with newer, more energy efficient ones, the 70% clean energy goal may not be so unrealistic after all.
For software developers in Hawaii, there are even a number of unique opportunities to contribute to the energy solution. For example, the Hawaii Natural Energy Institute (HNEI) implements various ways to collect energy usage data from consumers through installed devices in households which can send energy use information to HNEI as well as give users a more tangible reading on their own energy consumption. This allows HNEI to study not only energy usage in these households but also energy usage behaviors before and after the consumers are aware of their own consumption rates. Naturally, with all this collected data, software development expertise is then employed in the analyses and parsing of the information which may even be represented as graphical interpretations of energy usage that consumers can then access. Another opportunity for computer scientists interested in energy studies appears at Hawaii's flagship university campus as it undergoes renovations and seeks to improve energy efficiency in its buildings through wireless sensors streaming energy usage data to a central computer. Again, with all the data collected, there are opportunities for analyses and manipulation of the raw data into something coherent and useful. Likewise, with the rise in energy consciousness come more energy efficient homes and again, data collected from these homes provide unique opportunities for software engineers to help analyze it.
Even though being a mayor of Sim City might seem like a trivial task, the underlying lesson that one can take away from it is clear. Whether it's a computer game or reality, the need for better energy solutions exists and is more evident in the state of Hawaii than perhaps anywhere else in the country. Hopefully by now you also see why a software engineer might be interested in being part of the solution on a deeper level than simply the fact that their career choice relies on energy because, in fact, there are more than ample opportunities out there to challenge us and put our expertise to practical use.
 |
| Look at this thing ruining your citizens' day |
At this point, you might be wondering what this post is doing on my software engineering blog. The answer is two-fold. First, since you are reading this blog at all, you are on an electronic device with access to the internet and the concept of power and energy should at least interest you on a superficial level even if it's not a lingering thought in the back of your mind all the time. Secondly, this is my blog and I can write whatever I see fit on here!
Having said all that, allow me to express something of importance to this entry. I live in Hawaii. This is important because Hawaii, unlike most places in the country, faces a very unique situation when it comes to energy. Due to its relatively isolated positioning in the world, Hawaii is heavily dependent on importing oil in order to keep our lights on. What this amounts to is that our residential electricity prices lead the nation by nearly triple. It's clear to see now how the energy challenges that we face in Hawaii might concern a software engineer whose livelihood depends on having consistent access to electricity. Even putting aside the obvious reliance of a software engineer upon power, it is also important to take note of Hawaii's unique challenges from a concerned citizen's perspective.
In addressing Hawaii's energy situation, it is good to first understand that energy is the measure of power used over a given amount time. In other words, a light bulb using a little power that's on for a long time can end up using the same, or even more, energy as a microwave that uses a lot of power but is only on for a few minutes in the day. Energy consumption is therefore what we must keep our eye on even more so than how must power a device requires to operate.
Energy conservation is a concern all over the world, but it is of particular importance in Hawaii. As I mentioned previously, Hawaii's reliance on oil for energy generation raises the costs of energy quite a bit. Additionally, due to the fact that Hawaii is geographically isolated from the rest of the world, its energy grid is isolated as well. This means that energy can't be freely transferred from grid to grid as needed as is the case in the continental U.S., not to mention the fact that Hawaii's power plants need to be smaller and therefore less efficient as well. What all of this amounts to is essentially that traditional methods of power generation aren't nearly as effective in Hawaii as they are elsewhere, yet it is very difficult to uproot these practices and adopt new methods as well.
Granted, it is often difficult to buck traditions and change is often slow and sometimes painful. Fortunately, there is and should be optimism for the energy situation in Hawaii because, along with its unique energy problems, Hawaii also has unique potential solutions. For example, although Hawaii's small size results in inefficient power grids, the same small size means a more modest need for energy overall compared to larger metropolitan areas. Furthermore, Hawaii is uniquely primed for numerous sources of renewable energy as well, including wind, wave, solar, ocean thermal and geothermal. Couple the potential for renewable energy with the desirability for Hawaii and its people to ditch the reliance on imported oil and there certainly exists reason for optimism.
Of course, better people than I have thought of these things before me and it is through these people that I am able to convey some of these thoughts to you today. It would especially be of surprise to me if any of these concepts were lost on the folks at the Hawaii Clean Energy Initiative (HCEI). The HCEI ambitiously plans for Hawaii to achieve 70% clean energy by the year 2030. There is probably initially quite a bit of skepticism for this number upon hearing it. However, the plans call for 40% of the energy to be from renewable sources and also reducing our need for energy by 30% thus enabling the 40% of renewable energy to effectively be 70% of the overall energy requirements. As people grow ever more conscious of the growing need for an energy solution and become more efficient with their energy use, even by simply replacing light bulbs with newer, more energy efficient ones, the 70% clean energy goal may not be so unrealistic after all.
For software developers in Hawaii, there are even a number of unique opportunities to contribute to the energy solution. For example, the Hawaii Natural Energy Institute (HNEI) implements various ways to collect energy usage data from consumers through installed devices in households which can send energy use information to HNEI as well as give users a more tangible reading on their own energy consumption. This allows HNEI to study not only energy usage in these households but also energy usage behaviors before and after the consumers are aware of their own consumption rates. Naturally, with all this collected data, software development expertise is then employed in the analyses and parsing of the information which may even be represented as graphical interpretations of energy usage that consumers can then access. Another opportunity for computer scientists interested in energy studies appears at Hawaii's flagship university campus as it undergoes renovations and seeks to improve energy efficiency in its buildings through wireless sensors streaming energy usage data to a central computer. Again, with all the data collected, there are opportunities for analyses and manipulation of the raw data into something coherent and useful. Likewise, with the rise in energy consciousness come more energy efficient homes and again, data collected from these homes provide unique opportunities for software engineers to help analyze it.
Even though being a mayor of Sim City might seem like a trivial task, the underlying lesson that one can take away from it is clear. Whether it's a computer game or reality, the need for better energy solutions exists and is more evident in the state of Hawaii than perhaps anywhere else in the country. Hopefully by now you also see why a software engineer might be interested in being part of the solution on a deeper level than simply the fact that their career choice relies on energy because, in fact, there are more than ample opportunities out there to challenge us and put our expertise to practical use.
Subscribe to:
Posts (Atom)