In my last post, I intended to bring to everyone's attention the energy situation in Hawaii and how, in the midst of a unique problem, there are unique solutions. I also touched upon the fact that there exist numerous opportunities for a software developer to contribute as part of these solutions, especially in terms of data analyses.
As a concept, preparing a system for energy data analysis seems like a trivial task. Install a power meter, read that meter at some interval, store that data into a database and then query the database. However, when the time comes for implementation, it becomes a very involving task indeed. Luckily, we live in a time where open source applications are numerous and it's often easy to find an open source project that suits your needs, even if only partially so. Shortly after my previous entry, I learned of an open source web service which collects electricity data and stores it in a database. The service, aptly named WattDepot, even features a robust API for accessing the collected data. WattDepot can even be installed locally for experimentation or simulations and is capable of near real-time feedback. Of course, the best part of WattDepot might be that it's open source, and thus free to use for anyone who is interested in taking advantage of its many features.
But, as with almost all open source projects, there is certainly a bit of a learning curve involved. When I first tried my hand at creating a Robocode robot, there was quite a bit of trial and error as well as a heavy dose of perusing the Robocode API. WattDepot was no different. Although I had access to a WattDepot server which was already set up, and I knew that I had a wealth of predefined tools to help me interact with it, learning exactly how to do what I needed to do still took a bit of the proverbial banging of my head against the wall. Once again, in order to ease into a new system, I performed several katas to help familiarize myself.
1. Get sources. Print alphabetically.
One of the most basically things I learned quickly about WattDepot was that all the data came from sources. Each source had a variety of information associated with it, and one of the most useful bits of information is its name. In trying to alphabetize these sources, I tried to take advantage of the fact that the source names are Strings, and the String class's natural ordering happened to be, in a sense, alphabetical. Here, I "cheated" a bit and threw all the sources into a treeSet to shed the duties of sorting. Then the realization came that even if I were to sort the names, I wouldn't necessarily be able to keep each source's description associated with it. Luckily, Source objects in WattDepot have a compareTo method built in, and as fate would have it, the comparison was by name.
2. Get sources. Print from newest to oldest data.
The second thing I quickly learned about WattDepot was that there were SensorData objects associated with the sources. In order to get the timestamps of how recent the latest data was obtained from a particular source, I had to obtain the latest SensorData associated with each source. Here, I learned again how the compareTo method for this data type was implemented by throwing all the SensorData objects into a treeSet and printing them out. As luck would have it yet again, the objects were sorted by how "fresh" the data was, and printing out the name of the source the data came from was a simple exercise in parsing the String representation of the source.
3. Get sources. Print out their subsources. And their subsources. And their subsources...
With a heavy dose of String manipulation review in my mind now thanks to all the data parsing from the previous exercises, it wasn't particularly difficult to get split a source's subsources up and print them out. However, there was a bit of trouble here because in order to print out hierarchies, it made sense to make use of recursion. Unfortunately, upon investigation, all of the sources on this particular server had either only one level deep of subsources or none at all. What this means is that, even though I wrote my hierarchy printing code to be prepared for recursion, I was never able to fully test it out, since there is no recursion involved if there were no subsources that had its own subsources.
4. Get sources. Print energy consumed by that source... yesterday.
This particular kata or simple exercise took, by far, the longest of all. The goal was very similar to the first two exercises, yet the implementation had to be drastically different. When a query to the WattDepot database for the energy consumed by a particular source is made, the result is just the energy as a floating decimal number. In order to sort these numbers, I chose to put them all into an ArrayList. However, keep the numbers relevant, I needed a second ArrayList with the names to which the energy consumption data corresponded. Only then was I able to sort the lists and print out the results. This kata, however, took a longer time to implement because I hadn't had a lot of experience manipulating Calendar objects, but it turned out to be a surprisingly powerful and useful tool.
5. Get sources. Print highest recorded power per source... yesterday.
With my new found understanding of Calendars, and having already set the foundations for filling two lists which correspond to one another in order to sort them, this kata took a bit shorter to implement. However, due to the fact that we needed to find the highest recorded power by each source, we had to sequentially query the database a number of times per source in order to compare and find the highest recorded number for each source. This not only added more loops, decreasing the performance of the application, but added a lot more individual queries to the server, and overall, the execution of this particular kata was almost excruciatingly slow. In order to expedite the process, I decreased the query intervals to a mere two queries per source and when I was satisfied that the code was serviceable, I reverted back to shorter intervals between queries. Unfortunately, much like the conundrum between exhaustive and practical testing, I was only able run practical tests with longer intervals so as to avoid waiting for long periods of time just to test the execution.
6. Get sources (I see a pattern here). Print the average energy used in the last two Mondays.
Given the last two katas, the most trouble this one gave me was actually in the date manipulation. I found that, if you tried to set the Calendar object's day of the week to Monday, you will end up getting the upcoming Monday's date. What I needed was the past Monday's date. After wracking my brain briefly, I ended up going with the brute force method of the good ol' switch statement. If today is Monday, go back seven days. Otherwise, if today is Tuesday, go back one day and so forth until Sunday, where we go back six days. No, this was not the most elegant solution, but given how little I still understand about how Calendar objects fully function, it was a workable solution.
Through these exercises, I got to understand WattDepot a lot more and, as much as I hate to admit it, I learned a lot about Java that I may not have otherwise learned. While it is on a crude level so far, a lot of the energy data manipulation I learned from these exercises involved sorting data and formatting the output. This is deceptively important, however, because a big part of studying data involves presenting the data in ways that the data can be visualized and trends and patterns might be identified. I was able to complete these six katas to my satisfaction but I unfortunately neglected to jot down the specific amounts of times each one took to implement because I tend to do my coding with breaks interjected. Ultimately, however, it isn't the amount of time devoted to learning a craft that matters, but what you are able to derive out of the experience instead. In this case, I learned a lot more about WattDepot and how to manipulate the information I can attain from interacting with it by sitting down and devoting time to learning and I truly hope that I can inspire you to do the same. Even if it's not WattDepot, or even computer science... sit down and take the time to learn something today! After all, if I hadn't put in the time and effort, I'd definitely still be asking "what depot?"
No comments:
Post a Comment