Erik on Software

BigDataCamp Chicago (April 21, 2012)

Personal notes from BigDataCamp Chicago attended yesterday at Morningstar. The first BigDataCamp Chicago, for which I registrered but could not attend, was held on November 19, 2011. This time around, the title for this event was amended to include "Open Data", to emphasize working with data publicly made available by the government to solve problems in the public sector.

This past week, I already had the chance to browse a number of data stores made available by the City of Chicago and other government entities, mainly because I was looking for big data sets with which to work while ramping up on R Language fundamentals, and so I decided not to attend the sessions specifically focusing on the "Open Data" topic.

It so happens that I found out about this unconference through a nonprofit consulting firm, MCIC, for which I had performed some pro bono consulting work a couple years ago. The company had sponsored a competition this past year to solve public sector problems with open data, and there were connections between that competition and the upcoming Big Data Week this year.

The MC for the opening segment was Jonathan Seidman, formerly with Orbitz and now with Cloudera (he noted that Cloudera is "hiring like crazy"). Orbitz has been a heavy user of Hadoop for the last couple years. Seidman noted that there are "a ton of applications" in this space. For example, application of Hadoop to the health sciences and government.

RelayHealth applied to the connectivity between providers and payers, which involves the transferring of large volumes of eligibility and claims data, and realized that Hadoop would help. They were required to archive 5 years of data, and wanted to expose inefficiencies in processing claims etc.

First Life Research has a website that merges health care with social media and big data, aggregating information from patients from real life experiences, analyzing "billions of discussions" to discover real world treatments.

Explorys integrates clinical, financial, and operational data to improve health outcomes and reduce medical costs. Hadoop forms the base of their "DataGrid" big data platform. Pacific Northwest National Laboratory uses tools such as Hadoop and HBase for bioinformatics.

USA Search, part of the general services administration for the U.S. government, is a hosted search service for usa.gov and over 500 other government websites. They started with MySQL but later turned to Hadoop and Hive to provide a low cost and scalable solution.

Ted Dunning, currently MapR Technologies Chief Application Architect, shared the opening address alongside Jonathan Seidman. Speaking to some slides to which I hope to obtain access in the near future (Ted sat by me during lunch and mentioned he would post in the near future), Ted mentioned that that big companies would be adopting big data technologies such as Hadoop if there were availability or cost concerns. But what ends up being a more common scenario is that backwards adoption is taking place.

What Ted is seeing is adoption by startups prior to large firms. While big data technologies are being applied simultaneously by both the small and large, the conventional answer in terms of whether to adopt these technologies is the amount of data involved. The old school experiences exponential scaling, whereas big data has linear scaling in terms of cost.

To get value, diminishing returns well always be experienced at the outset. Old school thinking tends to think that the cost curve is steep, but big data makes cost linear, driving down the cost coefficient. The cost curve tends to follow the value curve. Algorithms have to scale horizontally, on commodity hardware. "What can you do with it, and how? This is what we'll be discussing. We've got a revolution on our hands."

---

Following the philosophy of an unconference, a number of session ideas were proposed, and attendees (about 150) voted as to which should be turned into 4 rounds of sessions.

Proposals, with approximate number of votes for each according to MC:

• Hadoop cluster recommended hardware configuration (15)
• open data decision making and tradeoffs (12)
• deidentification of data (40)
• using social media analytics to make chicago a better place to live (50)
• big data visualization (60)
• open data for health care and quality (?)
• moving data practice from B2B space to customer space (10)
• Hadoop, HBase, MongoDB in the OLTP space (35)
• scaling from legacy data warehouse to Hadoop in agile manner (10)
• big data use cases (15)
• intro to R and working with big data (a lot)
• mega learning with Mahout (50)
• opening up 311 for web applications (?)

Following lunch, the schedule was as follows (emailed by the organizers). The sessions I attended are in bold. Note that since this was an unconference, the session focus did not necessarily stay on target. Attendees of each session provided the direction of each discussion.

1:30 pm 
Auditorium #1 - Intro to Big Data & Use Cases
Training Room #2 - De-identification - Privacy for Public Data
Training Room #3 - Hardware Req'd for Hadoop Cluster

2:15 pm 
Auditorium #1 - Intro to Hadoop & R & Big Data
Training Room #2 - Open Gov: Data for Civic Applications (ex: Healthcare)
Training Room #3 - Prepare your company for Big Data (BTC)

3:00 BREAK

3:15 pm 
Auditorium #1 - Mega Learning w/ Mahout (esp K-NN)
Training Room #2 - Social Media Analytics
Training Room #3 - Scaling from Legacy Data Warehouse Vendors to Hadoop

4:00 pm
Auditorium #1 - HBase for Realtime
Training Room #2 - Big Data Visualizations< br />Training Room #3 - n/a

4:45 pm Wrap-up & BEER!!

---

Training Room #3 - Hardware Req'd for Hadoop Cluster

This session began by posing an example scenario in which 40tb of data exists in storage, to which 1tb of data is added each month. How is hardware selected to support this volume? Ted got up and mentioned that 10 times this workload is also very common, so we should really discuss as well.

The discussion then turned to on premise, private and public cloud storage. What these different types of implementations really mean. Power consumption etc. Ted mentioned that the cloud is probably not the route to go. The cloud is cheaper with regard to elasticity. So we focused on the data center.

Working through the rack units currently available in the marketplace, Ted explained that with 1U servers, most purchase SATA drives because these are the lowest cost. 7500rpm, 2tb. 1 or 2 CPUs. 1x6 cores, 2x6 cores. 4gb/core.

2U servers are the next level up. 12x2tb. 1 or 2 sockets. As a side note, Ted mentioned that RAID is typically not recommended in the big data space. 4U servers are 20x2tb or 36x2tb. One 4U server will be released on the market that has 80x3tb (240tb) drives in one chassy.

According to Ted, community Hadoop does not do a lot of drives well. MapR Hadoop gets around these issues. There is a power limit for each rack, so with community Hadoop you can only partially fill racks, wasting money. MapR Hadoop can make use of 4U servers.

Dell, HP etc - everyone - offers 1U and 2U servers. 4U 20x2tb is offered by several, and 4U 36x2tb is only offered by one vendor now. Relative cost for 1U, 2U, and 4U is about $4k, $6k, and $12k, respectively. The last, 4U, is the most cost effective.

Community Hadoop gets bottled at CPU level, not the controller level, so more disk space is needed for community Hadoop. Each step of merging needs to go to disk, because Hadoop has to deal with failures, so I/O speed makes a huge difference. With Hadoop, data is typically triplicated.

Hadoop can only do about 400mb/sec on 1gb/sec interface, and about 720mb/sec with 100gb/sec interface. Co-occurrence accounting. Most machine learning is about counting. Switches are expensive, about $4k each. MapR has broken some switches recently. Ted trusts Cisco switches the most.

Aggregration jobs are very common, with 90% to 95% of activity associated with local reads. With virtualization, Ted cares first about what is gained rather than what it costs. Ted commented that since he sees no advantages in using virtualization for Hadoop, he does not care about the costs, and joked that not having sys admins "pissed at you" is a benefit that often outweighs costs.

Between 4gb and 8gb per core of memory is recommended -  this does not vary much. Ted recounted someone commenting in a conversation once that he did not care about losing data in Hadoop, and he "got demolished" by the others attending the meeting.

After Ted commented that "Hadoop is not real time", one attendee of this session mentioned that disaster recovery in Europe and the UK is a requirement. Ted followed up by remarking that human error or human error by proxy (the application) is usually the issue.

The attendee joked that from a US perspective, if there is a nuclear attack or some other disaster, who really cares about the data? Ted mentioned that it is fairly standard to send the same data to two data centers. Then the processing can start on one data center, and mirrored on the other data center. Ad hoc work can be performed at one data center, but in light of disaster, ad hoc work would stop. The serious cost is resynchronization of data.

---

Auditorium #1 - Intro to Hadoop & R & Big Data

A raise of hands indicated that session attendees were split about evenly across beginner, intermediate, and advanced users of the R language. The first moderator of this session mentioned Google's notice that both the best and worst aspects of R is that it is a language built by statisticians for statisticians.

Bell Labs gave us the S language, and R arose from S. S ended up getting licensed out by Bell Labs as S+, but it failed. The rise of R is a case where an open source technology outperformed the commercial version of the product.

R came out of a version created in Ausralia for the Mac, and adoption grew from there. R really lives by having its data in memory, so when it comes to big data, you need to have a plan.

R is open source, free, and extensible. What makes R popular is CRAN, which forces documentation, QA checks etc. There are now about 3,600 packages that can be plugged into R core. About 10 years ago there were less than 100 available packages.

One of the session moderators noted that they are associated with the "finance" and "high performance" packages, having authored about a dozen of them. While big data falls into this latter category, the "high performance" packages have broader scope than just big data.

After it was noted that R needs GCC, I mentioned some of the development efforts I have been investigating of late, such as Renjin, which is built on top of the JVM. The session moderator mentioned that there have been many efforts over the years to work on similar projects - porting to different platforms - but they never seem to get 100% complete, so he had no opinion but admitted that he does not follow any of these efforts closely.

The R language is single threaded, and according to the session moderator "this will not change". Following up on this discussion, he mentioned that he likes to combine C++ and R with his development efforts.

The size of the R core engine is not really big, so installing it on multiple nodes is not a bad thing to do. R is pushing 40 years, so it is impressive that it is keeping up.

The second moderator of this session (the author of Parallel R, who goes by the name "Q", pictured to the right in the above photo) declared that Hadoop does not really do anything new. Cluster computing has been around a long time. However, Hadoop is friendly and talks to anything - Java APIs, C++ APIs etc - which is one reason it became so popular.

Hadoop is a general toolkit that you can pretty much use for anything. R is not good with big data, which is why working with Hadoop is such a natural match. The gist of MapReduce is that it operates on the principle of divide and conquer. Taking data and breaking it down into manageable chunks.

What makes MapReduce so powerful is that all of the mapping and all of the reducing can be handled in parallel. One good application of blending Hadoop and R is when certain calculations need to be performed over and over. Unfortunately, I missed what the moderator said about the second application.

If you do not have a problem that falls in one of these two categories, it does not make sense to use Hadoop, and you will waste a lot of money.

An R package called Segue was written by someone who did a lot of financial models (J.D. Long). Segue sends data to the cloud, which may be a big drawback for some. An RHadoop package called rmr permits the running of MapReduce jobs from R. The biggest drawback with this package is that you need to understand both R and MapReduce.

According to the session moderator, Hadoop Streaming is "complicated and not suitable for exploratory work". (Note that I stumbled upon a news release in this space involving a partnership between MapR Technologies and Informatica while looking for this link.) One of the final notes made was that MapReduce may not work for you if you need to have everything in memory, because the first thing MapReduce does it split up your job.

---

Auditorium #1 - Mega Learning w/ Mahout (esp K-NN)

Ted started with a story. He had a customer who thought that the k-nearest neighbor (KNN) model might work for them, so they ran a test than ran 3,000 queries against 50,000 reference accounts. But their results were 1,000,000 times too slow. Ted worked on some rapid development with the customer, resulting in 2 times too slow.

They used Mahout to get these results. Mahout is an Apache project that enables scalable learning. Ted explained that you can make a new type of matrix on the cuff, which is something difficult to do with most math libraries. The approximate KNN search that was created can perform about 2m queries in an hour or so.

Mahout does 3 things: (1) performs recommendations with mature production quality (AOL, Amazon etc use it, and a company is currently attempting to commercialize its functionality), (2) performs supervised learning and classification - you know a bunch of answers, and it is replicated (with 100k training data use R, and with 100m training data Mahout is very good, but 1m is borderline), and (3) provides clustering.

According to Ted, Mahout is very fast for what it does. It provides a collections library that is not GPL, and is "a very active project, a warm, welcoming community". Ted is a committer on the project, and one of the attendees of this session commented that Ted responds very quickly on the Mahout distribution list.

Unlike Mahout, which uses a single pass algorithm, "Hadoop is like molasses starting a program" because it uses 3 passes. A k-means is done to solve KNN. What makes it fast is the KNN that is performed within the k-means. A cluster is a surrogate to the probability distribution.

Ted mentioned that most of Mahout works without Hadoop, although some parts need it. One of the session attendees asked why Mahout does not perform a specific function (they were hard to understand), and Ted said that it was a good idea and asked that they submit to JIRA so that it can be implemented.

---

Auditorium #1 - HBase for Realtime

Norm Hanson, team lead at Dotomi, started this session by discussing some of the work with which he is currently involved at work (he shared a portion of this session with me and a couple other attendees earlier in the day during lunch).

Norm is seeking to replace a "super high availability" OLTP database. PostgreSQL is currently being used. The data is key-value based so the database is using a single index path. But the business, and therefore the data, is growing. And they cannot take the database offline to grow the data store. He surmised that they need "some type of horizontal scaling mechanism".

Last September, they had to buy more hardware and then redistribute the data, creating new data partitions etc. He wants technology that is horizontally scalable out-of-the-box. One of the session attendees asked why they do not just use MongoDB, and Norm joked, motioning to Ted, that "the MapR guys are trying to sell me MapR".

Norm comes from an Oracle database background, where 5-9's is fairly standard. He figured that one solution might be to break up the data into pods, where certain portions of the data are divided from each other. They already have a Cassandra cluster, but it does not scale under heavy loads.

At this point, Ted mentioned that HBase has the concept of coprocessors, and mentioned that he will talk about realtime aggregations at the next Hadoop User's Group. Essentially, MapReduce jobs can be run against HBase at very high speeds.

Norm remarked that disks always fail, and that you need to test for disaster recovery, period. He also thinks that for OLTP these technologies are going to replace the Oracles of the world, and commented that he does not want anyone running MapReduce jobs against his OLTP data store.

Ted mentioned that HBase just looks at data like BLOBs, and that community support is a good reason to go with HBase, compared to other technologies. He has seen places where MongoDB is the right way to go, but just for small scale efforts usually associated with startups that need a good out-of-the-box experience.

According to Ted, Cassandra is "absolutely right" for Netflix, which has serious constraints on partitioning. Cassandra keeps track of a lot of noncritical data, such as the last spot in a movie that is being watched. HBase has a strong consistency model, which makes it bad for partitioning.

HBase insert rate is high. There is no order of magnitude difference in terms of speed between HBase, MongoDB, and Cassandra. HBase usually has strong service agreements. Some load isolation is needed. You can do tricks with HBase that you cannot do with the others. One reason for this is that you can load 30m rows per second with HBase. Ted sited an example of one company that completely rebuilds its database on a daily basis.

Do not perform joins or multirow updates with HBase. A high availability HBase name node is a must. CDH4 and MapR have high availability name nodes. Many-to-many relationships may be the wrong match for HBase. HBase is not a relational database, so you can "do outrageous things". HBase has the concept of a multiget with a regular expression, returning all rows back with one read.

Ted emphasized that the key design decision here is the fat table versus the skinny table. The skinny table, for example, has one row per person, person ID. The fat table has one column for everything - person, product, etc. Norm commented that he is considering use of Hive to import to Green Plumb.

Helpful Votes: Part 5

This past Thursday (May 24, 2012), my Amazon reviews hit two milestones: 1500 cast votes and a breaking of the Top Review Ranking of 2000 for the first time in almost 2 years, bringing it to 1967. Over the last 2 months, my ranking shot up about 500 spots due to the incredible response I have received for my review of Madeleine Albright's "Prague Winter: A Personal Story of Remembrance and War, 1937-1948". The last time a review of mine generated anything resembling this interest was my review of Andrew Wakefield's "Callous Disregard: Autism and Vaccines - The Truth Behind a Tragedy" back in August 2010, bringing my ranking to just over 1950 (my highest ever) within weeks after publishing my review.

Since the time of this last breakthrough review, my ranking has consistently fluctuated between 2000 and 2500, and it was the experience of writing that last review that showed clearly the difficulty of writing reviews that appeal to a wide audience. The fact is that negative votes on this Wakefield review started pouring in after the book started gaining attention in the media, and Amazon visitors cast their votes due to political reasons rather than whether they actually found the review helpful. As I have mentioned in past blog posts in this series, this dilemma is really the true obstacle in writing reviews, at least on Amazon.

Casting a helpful vote for an Amazon review is intended to indicate that the review swayed the reader of the review in a direction that provided input to their decision to read the book, regardless of the direction of their decision to read. Unfortunately, it is common that readers vote based on whether they agree with the book, not whether the reviewer provided input to their decision making process. Amazon is an immensely popular store to purchase books, and the review system that it provides is one of its best features, but the nature of this system being open to the public unfortunately cannot prevent this abuse.

This situation just comes with the territory. Interestingly enough, just this past month an author who had reached out to me in September 2010 to write a review for their book on software architecture, which has since had its third printing, mentioned the following after first thanking me again for writing such a good review:

I do not understand Amazon reviews/reviewers.  For example, only 1 in 3 people found your review helpful?  Really?  I found it to be a nice mix of spelling out where you're coming from, what you liked and disliked, and advice for other readers.  For me, that's an ideal review.

Personally, I find such a compliment the utmost reward for helping readers decide what to add to their reading lists, especially in the technical space. I have mentioned in previous installments of this series of blog entries that I do not write reviews with a goal of reaching a high Top Reviewer Ranking. Quite frankly, I realize that I do not have enough time to read the significant number of books that would be required to reach lofty rankings. I have a policy to always actually read all of the books about which I choose to write, which might detract from my rank potential, but in my opinion keeps my integrity on this matter sound.

New Book Review: "HBase: The Definitive Guide"

New book review for  HBase: The Definitive Guide, by Lars George, O'Reilly, 2011, reposted here:

This text is amongst the few books I have read in my career which not only serves as a great introduction to a technology, but also provides significant content for both advanced users and those who wish to understand what is under the covers in order to understand both benefits and drawbacks before making a decision in terms of adoption. While I was not familiar with the author, Lars George, until reading this text, the foreward to what the author shares here offers some street cred. "Of all those who have contributed to HBase over the years, it is poetic justice that Lars is the one to write this book. Lars was always dogging HBase contributors that the documentation needed to be better if we hoped to gain broader adoption. Everyone agreed, nodded their heads in ascent, amen'd, and went back to coding. So Lars started writing critical how-tos and architectural descriptions inbetween jobs and his intra-European travels as unofficial HBase European ambassador. His Lineland blogs on HBase gave the best description, outside of the source, of how HBase worked, and at a few critical junctures, carried the community across awkward transitions (e.g. an important blog explained the labyrinthian HBase build during the brief period we thought an Ivy-based build to be a 'good idea'). His luscious diagrams were poached by one and all wherever an HBase presentation was given."

At least one other reviewer here commented that it was hard for them to understand the purpose of this book, because of its broad scope. But in my opinion its broad scope is what makes it so valuable, at least to the architect. The potential reader just needs to keep in mind that they need to understand what they wish to gain from reading a text on HBase. For example, if the potential reader simply wants to understand how to use the APIs that HBase provides, there may be no need to understand HBase architecture and so the chapters that the author provides on related material may not be applicable at the outset. It is difficult to argue for a sequential reading of this text from beginning to end, although the potential reader also needs to keep in mind the importance of minimal understanding of the programmatic connections with the architecture, in which case a cursory overview of some of the more advanced material is highly recommended. In contrast, as an architect who wants to understand as many of the benefits, drawbacks, and implementation details as possible, especially when considering my extensive experience with relational databases such as Oracle, I appreciate all of the detail that George has to offer here, and would not have it any other way. While I have not yet reviewed his blog yet to any great extent, based on his street cred and the abundance of material in this book, I do not think it is a stretch to akin him to Tom Kyte of "Ask Tom" fame within the Oracle space.

After an introduction that provides discussions on big data, column-oriented databases, problems with relational database systems, nonrelational database systems, and an HBase architectural overview (all within Chapter 1), George quickly moves forward to a chapter on HBase installation (Chapter 2), followed by discussions of native Java APIs (Chapters 3, 4, and 5), available clients (Chapter 6), and integration with Hadoop's MapReduce framework (Chapter 7). The remainder of the text delves into more advanced topics, beginning with HBase architecture (Chapter 8), followed by design topics critical to taking full advantage of HBase architecture (Chapter 9), as well as discussions on cluster monitoring (Chapter 10), performance tuning (chapter 11), and cluster administration (chapter 12). As a general rule, George does a great job at introducing topics in an order that makes sense without frequently referring to chapters later in the book, which is a pet peeve of mine with other technical books. One of the many aspects I especially appreciated about this text is that even in chapters that cover content that might be seen as high-level content by other architects, George is not afraid to get down in the trenches at the programmatic level, even in the preface to the book, in which the author explains how to build the examples that he provides in his Git repository.

Please be aware that even though the author discusses features such as coprocessors that have been included in subsequent releases (at the time of writing this review, version 0.92.1 has been made available as a stable release), the author does make note in Chapter 2 that "the current version of HBase will only run on Hadoop 0.20.x. It will not run on Hadoop 0.21.x (nor 0.22.x) as of this writing. HBase may lose data in a catastrophic event unless it is running on an HDFS that has durable sync support. Hadoop 0.20.2 and Hadoop 0.20.203.0 do not have this support. Currently, only the branch-0.20-append branch has this attribute. No official releases have been made from this branch up to now, so you will have to build your own Hadoop from the tip of this branch." The author does bring up the fact that Cloudera's CDH3 has already applied the patches associated with durable sync, if building Hadoop is not your cup of tea, but keep in mind that Cloudera as of less than two weeks ago has already released CDH4 Beta 2. Anyone interested in HBase needs to keep in mind that this space is moving along quickly, so plan to get started with this book and move forward with the abundant material that is increasingly being made available online. That said, George provides a wealth of knowledge that I do not expect to be surpassed at any time in the foreseeable future by any other publication.

Google Analytics: 2.5 Years

In looking at the calendar, I realized that it was about time that I submit another post for a periodic series focused on Google Analytics, which I put in place for this blog about 2.5 years ago (October 2009), although the history of this blog stretches back another 6 months or so. Visitors from an additional 4 countries have visited this blog since the last update, bringing the number of countries of origin to a total of 117 (refer to the posts I submitted during this time period to see how traffic has grown).

Visitors from the top 10 countries of origin came from the United States (45.6%), the United Kingdom (4.2%), Canada (3.9%), India (3.8%), Germany (3.4%), Australia (2.7%), Brazil (2.5%), France (2.5%), Spain (1.9%), and Netherlands (1.9%). The top 5 countries of origin have remained static. From another perspective, visits from the top 10 cities originated in (1) New York City, (2) London, (3) Chicago, (4) Moscow, (5) Bangalore, (6) Singapore, (7) Sydney, (8) San Francisco, (9) Melbourne, and (10) Los Angeles.

The top 15 blog pages visited here are as follows: (1) Anima Sana in Corpore Sano: Part 1, (2) Copycat Advertising Campaign, (3) Liquibase in the Enterprise: Part 2, (4) Switching from ASICS to Brooks, (5) Liquibase in the Enterprise: Part 1, (6) Liquibase in the Enterprise: Part 3, (7) Global Leadership Summit 2010 - Day 1: Andy Stanley, (8) Liquibase in the Enterprise: Part 4, (9) New Book Review: "Manage Your Project Portfolio", (10) RSNA 2010: Quality Improvement? - MRI Workflow, (11) Global Leadership Summit 2010 - Day 1: Dr. Peter Zhao Xiao, (12) Treadmill Maintenance, (13) SpringOne 2GX 2010 - Day 2 Keynote, (14) Anima Sana in Corpore Sano: Part 2, and (15) "Hello Android": Part 4.

Some musical chairs over the last 6 months to some degree in the lower half of these top posts, with two notable exceptions. The fourth in my ongoing series on Liquibase in the enterprise moved up three spots, and the first from my "Hello Android" series made an entrance.

In reviewing these metrics from Google Analytics, I am warming up to the new version. In spending some more time exploring the new user interface, it seems to be much quicker than the old version, and the navigation in my opinion is better. Although the theory is that over time the composition of visitor countries of origin will start to resemble global web traffic in general, I still find this metric interesting, especially since this blog is technical in nature.

Drilling down in the map overlay is much more intuitive, and I like the fact that cities of origin with as little as 1 visitor are now displayed, as opposed to the old version. For example, with the region map overlay of Germany, darker shades of green continue to depict more visitors. Here, we see German states Nordrhein-Westfalen and Baden-Württemberg provide the most visitors during the time period used for the following map:

The hover display for Berlin signifies that the pointer for the user interface is pointed at Berlin. Switching to the city map overlay, darker shades of orange continue to depict more visitors. Here, we see German city Frankfurt am Main provides the most visitors during the time period used for the following map:

The hover display for Berlin now signifies 15 visits for the time period rather than 16. Panning across Berlin, it is apparent that while Berlin is the capital city of Germany, the nearby capital city of the German state Brandenburg, also part of the Berlin/Brandenburg Metropolitan Region, was actually responsible for one of these visits.

Based on my experience with the old version of Google Analytics, this visit from Potsdam would not have been displayed. The new city map overlay is also more effective in the sense that cities that have been visited less frequently are always depicted on top of cities that have been visited more frequently, as to not crowd out cities of origin with fewer visitors, which was the case with the old version of Google Analytics, as well.

 

While as a software professional I am routinely cognizant of the fact that multiple browser products exist on the market, each time I revisit Google Analytics for this blog I am reminded that there are many browsers that my colleagues and I do not target for development. For another example time period used to produce the above output, it is apparent that 17 different browsers were used to access this blog.

At the top end of frequency, browsers Firefox, Chrome, Internet Explorer, and Safari are readily recognizable. But what about browsers such as Internet Explorer with Chrome Frame, SeaMonkey, Camino, and RockMelt? So I do recognize that Internet Explorer with Chrome Frame makes use of a Chrome plug-in for Internet Explorer, but the metrics above serve as a reminder how little it is actually being used.

According to the release notes for SeaMonkey, the browser has enjoyed regular releases since 2001, with the latest version 2.9 beta being released just 2 days ago. Specifically designed for the Mac OS X operating system, Camino has also been released on a regular basis since its initial release as Chimera back in 2002. And RockMelt is a social media web browser based on Chromium just like Chrome.

New Book Review: "The Wrong Answer Faster"

New book review for The Wrong Answer Faster: The Inside Story of Making the Machine that Trades Trillions, by Michael Goodkin, Wiley, 2012, reposted here:

A coming of age story of a man and an industry. As a first generation American, a Chicago-area native, and a software professional, I find this memoir entertaining, inspirational, and at times enlightening. The author shares his early struggles as a student in the Chicago public school system that eventually bring success at the university level, and how the lessons learned of his early job experiences feed his later entrepreneurial efforts that result in two successful software companies. At times, Goodkin comes across as a bit unrealistic, perhaps because of what he decides to share, especially since he does leave out a significant portion of his life between these entrepreneurial efforts, but this book is amongst the few memoirs I have read that is also a true page-turner.

While my initial interest in this book was grounded in its presumed tie to software development, it is actually all of the other elements just described that will cause me to talk about it amongst colleagues. After all, it is not until Chapter 4 ("A Big Idea") that Goodkin starts discussing the trading specific ideas that lead to business success, and because the software described is proprietary he does not get into much detail about how it works, beyond sharing at a high level the underlying theories that help the reader understand how the algorithms already available to the public were until that time ignored by Wall Street. Many in the business world are all too familiar with good ideas initially rejected in the workplace, but the manner in which Goodkin walks the reader through his business development efforts is engaging.

In contrast to what some other reviewers have written, the author does admit, however little might be the case, that mistakes were made and how strategies needed to be reassessed and changed throughout the process of building these businesses. For example, he recognized the need to change his marketing approach after his attempts to move the business forward in parallel with a gradually maturing product were failing, and he ties together his previous experiences as founder of Counsel's Aide, a legal service he founded while attending Columbia University, and his failed attempts to interest the City of Chicago, extremely well in Chapter 8 ("Making the Impossible"): "I had once again mistaken a quality service for a marketing strategy. They had no interest in what I was selling because there was nothing in it for them."

"If they made the investment and it didn't pan out, they could lose their jobs. If it did work out, nobody would care. If anything, some of their bureaucratic brethren would be upset with them for having rocked the boat. Last, but certainly not least, nobody ever got a bonus for better performance. In a nutshell, pension fund managers were interested in their pensions." And later in Chapter 9 ("If I Were a Rich Man"), Goodkin remarks that "To my dismay, my sales abilities weren't keeping pace with my enhanced perception. Listening to myself talk, I began to understand why. At university, I had come to the conclusion that professors who used an inordinate amount of technical jargon did so to mask their shallow understanding about what they were talking about. I was no different."

In the Epilogue to this memoir, Goodkin shares the remarkable snowball effects of the efforts of he and the world-renowned professionals on his team: "From 1996 until 2011 the derivatives market grew from an annual $26 trillion to over $600 trillion. Numerix evolved accordingly. No longer solely dependent on accelerated Monte Carlo to provide the wrong answer faster, its product range has grown to include a full line of risk analytics providing the right answer. Indeed, Numerix has developed such a reputation for accuracy that when the supervisors of Lehman Brothers in bankruptcy required an independent appraisal of its massive derivatives portfolio, they turned to Numerix. With over 700 clients in about 25 countries, Numerix has become the global standard in risk analytics."

New Release: IBM DB2 10

While doing some research for a recent client, I came across a press release that announces the planned release of DB2 10 later this month (April 2012). At another previous client a couple years ago, I used DB2 9.5, and have not needed (or necessarily wanted) to look into DB2 again until now. Okay, so this is a proprietary relational database product. But it has some interesting features. I'm not going to write about all of these, but there are a few that strike a chord to some degree with me.

Starting with DB2 9.7, it sounds like IBM has been making it easier to migrate from Oracle stored procedures to DB2 stored procedures. I do not know about you, but I do not like stored procedures in general, and am architecturally wary of them because they provide a way to store an application layer within a data layer. At another recent client, there were hundreds of them (I do not remember the count anymore). One of the things I did over there was repackage them to make them easier to work with, especially with regard to database change management. Doing so did not make them more likeable, but as anyone working at that client can attest, they are definitely a constraint that needs to be reckoned with, and the same situation goes for other shops.

Unofficially, I am partial to the Oracle database. I have been for quite some time. So moving from Oracle to DB2 in itself does not sound like something I would ever want to do, if other options are viable. But in the case that a migration of Oracle stored procedures to DB2 is needed, it is good to know that DB2 now supposedly provides 98% compatibility at the source code level. I am sure there are a host of other incompatible factors beyond source code to keep in mind, such as those related to database tuning, that you will not find out about until you are further down the adoption path.

It is interesting that DB2 now offers bi-temporal features, especially after having just recently read one of the few texts available on the subject. The text is very weighty, and takes some time to get through. The authors discuss bi-temporal features that have not yet made it into the SQL standard, and propose their own solution. It is one of those books where you have to be able to follow along extremely closely from page to page, or you will get lost quickly. Similar to a series of math classes that you might have taken in college. The features that DB2 provides with regard to this feature set are together called "Time Travel Query".

IBM has a great slide presentation on this subject. The subtitle to this presentation mentions "DB2 for z/OS", because that version of the product was released in 2010. The upcoming release is for "DB2 for LUW" (Linux, Unix, and Windows), which has a different code base and is maintained by a different set of developers. This presentation sums up well what this release has to offer with regard to "Time Travel Query": Business Time, System Time, and Bi-temporal.

With Business Time, a pair of timestamps is set by the application: begin time (when the business deems the row valid) and end time (when the business deems that this row validity ends). This functionality permits queries against past, current, and future business time periods. System time is a pair of timestamps set by DB2 to every row: begin time (when the row was inserted to the database) and end time (when the row was modified or deleted). This functionality permits queries against past and current system time period. Bi-temporal includes both Business Time and System Time in each row.

One of the best aspects of this presentation is that the examples for Business Time and bi-temporal use an industry in which I have been working: healthcare insurance. Policies change over time, for example, and automating some of the functionality to keep track of these changes, while also permitting queries of data based on different time periods, all while minimizing risk with regard to whether a developer coded correctly, is a useful feature. And with System Time, the database moves historical records to separate database tables while not worrying about the need for developers to keep track of these different tables, since DB2 resolves for developers behind the scenes.

Another feature that peaked my interest is a new join method called zigzag join (ZZJOIN). Considering a star schema, DB2 provides two special join methods: semi-join with indexing (star join) and hub join (Cartesian join). Apparently, zigzag join has been introduced to expedite the processing of queries based on a star schema. This join method simultaneously filters the associated fact table on two or more dimension tables, skipping unproductive fact table probes in the process, and apparently has been shown to be an efficient method for querying large fact tables, significantly reducing I/O.

The interesting thing about this feature is that it is seemingly impossible to find any information on it, including syntax, even after searching DB2 manuals. The reference with the most substance, outside research papers, is located here. No examples. Presumably, more information will be available in conjunction with the upcoming LUW release, but information for the z/OS version is hard to come by as well, and it was released two years ago.

New Book Review: "Strategy and the Fat Smoker"

New book review for Strategy and the Fat Smoker: Doing What's Obvious But Not Easy, by David H. Maister, The Spangle Press, 2008, reposted here:

When I first saw this title, I was inclined to think that the author had intended the phrase "fat smoker" to be an oxymoron, because at least in my experience, there are very few smokers who are overweight. But after reading what Maister has to share in his most recent effort (see my reviews for some of his other texts, such as "Managing the Professional Service Firm", "First Among Equals: How to Manage a Group of Professionals", and "The Trusted Advisor"), I came to realize that he was describing himself.

In setting the stage, Maister notes in the first chapter that "many change efforts are based on the assumption that all you have to do is explain to people that their lives could be better, convince them that the goals are worth going for, and show them how to do it. But this assumption is patently false. If it were true, there would be no drug addicts, no alcoholics, or bad marriages in the world. 'Oh, I see, this behavior's not good for me? Ah well then, I'll stop, of course!' What nonsense!"

"And yet strategic plans and annual speeches by CEOs, managing partners, management consultants, and others continue to adopt this same useless structure: 'Look at how fabulous it would be if you were a fit, nonsmoking exerciser, David!' My usual response? 'True, but please shut up and go away.'" Later, the author follows up by drawing a parallel between the reasons for his sudden move to stop smoking, start exercising, and losing weight, and how change often finally comes about in the business world.

"We all know the main thing that works: a major crisis! If revenues drop off sharply, it's amazing how quickly businesses can act to deal with known inefficiencies and bad habits they could have tackled years ago." After discussing some of the aspects that are known about persuading people of change before the "heart attack" comes, the author brings up what will probably hit home to many business professionals, in professional services firms or elsewhere.

"Everyone in the organization has to decide if they want to try hard enough to sacrifice some of the present to achieve a better tomorrow. They may do so if they believe that the effort is serious. They definitely will not if they think those at the top are undecided or divided. Professional firms are afraid of this conclusion. They try to work around the skeptics, the nonbelievers, and the nonparticipants in their senior ranks, preferring to hold on to revenue volume rather than put together a senior team whose members are equally committed to reaching. That's fine, but you can't call it strategy."

As a consultant, one aspect that I appreciated about this book is that after an illustrious career and several popular books, Maister takes the risk that exposing some of his past and present faults will drive home the points that he attempts to make. And by and large, in my opinion he is successful doing so. In addition to the introductory chapters, I also especially enjoyed chapter 6 ("Do You Really Want Relationships?"), chapter 9 ("Tyrants, Energizers, and Cynics"), chapter 10 ("Why Most Training is Useless"), and chapter 19 ("Passion, People, and Principles").

This third chapter that I have listed, in my opinion, is one of the best chapters that Maister has ever written, and should be required reading for anyone already in the business world, or preparing to enter it, especially those interested in management. As he mentions in his introductory statements to this chapter, most companies use training as a business version of a "quick weight loss" program. "They hope that by training people in new things, they can quickly bring about behavioral changes among their employees. It almost never works." Why? Because training is an almost useless first step in a committed program for long-term change, especially when it is being used as a substitute for changes in managerial behavior.

New Book Review: "Hadoop in Action"

New book review for Hadoop in Action, by Chuck Lam, Manning Publications, 2010, reposted here:

After checking out reviews of what O'Reilly and Apress had to offer with regard to Hadoop, I ended up purchasing this book based on positive reviews, my past positive experiences with the Manning "In Action" series of texts in general, such as "Spring in Action" and "Java Persistence with Hibernate", formerly "Hibernate in Action" (see my reviews), and the fact that this book was the most recently published on the subject. In short, this text is well organized, and covers its focus on Hadoop well, but potential readers should be aware that about one-third of what Lam has to offer here are ancillary to Hadoop, and not with regard to Hadoop itself. Inclusion of the larger ecosystem within which Hadoop sits personally makes sense, and I do not think this aspect of the book detracts from what the author provides in any way.

The author provides a good introduction to Hadoop in the first three chapters, which includes a discussion on differences between Hadoop and traditional technologies in this space, such as relational databases, as well as a tour of Hadoop building blocks, working with files in the Hadoop Distributed File System (HDFS), and the anatomy of a MapReduce program. The next three chapters contain the bulk of the text, which focuses on writing MapReduce programs, and includes segments on chaining MapReduce jobs, joining data from different sources, creating a Bloom filter, and monitoring, debugging, and tuning.

The next two chapters offer a short cookbook in which the author presents 5 different general MapReduce techniques (Lam admits that specialized MapReduce techniques can be found rather easily by Googling, and that he does not intend this cookbook to be comprehensive in any way), as well as a chapter on managing Hadoop, followed by four chapters on running Hadoop in the cloud, brief introductions on programming with Pig (a Hadoop extension that provides a language called Pig Latin) and using Hive (a package built on top of Hadoop that provides a SQL-like language called HiveQL). and a chapter that discusses four Hadoop case studies from the New York Times, China Mobile, StumbleUpon, and IBM (the case study from IBM takes up about 50% of the discussion, and the case study from the New York times is less than a page).

Be aware that at the time of this review, this book was published over a year ago. One of the common complaints I read about what O'Reilly and Apress have to offer in this space is that their counterparts to this book cover older versions of Hadoop. In chapter 4, Lam mentions that "one of the main design goals driving toward Hadoop's major 1.0 release is a stable and extensible MapReduce API. As of this writing, version 0.20 is the latest release and is considered a bridge between the older API (that we use throughout this book) and this upcoming stable API. The 0.20 release supports the future API while maintaining backward-compatibility with the old one by marking it deprecated."

"Future releases after 0.20 will stop supporting the older API. As of this writing, we don't recommend jumping into the new API yet for a couple reasons: (1) Many of Hadoop's own library classes in 0.20 aren't written under the new API yet. You won't be able to use those classes if your MapReduce code uses the new API in 0.20. (2) Many still consider the most production-ready and stable version of Hadoop as of this writing to be 0.18.3. Some users are warming up to version 0.20, but we suggest you wait a little longer before going full production with it." The author follows up by writing that "by the time you read this the situation may be different. In this section we cover the changes the new API presents. Fortunately, almost all the changes affect only the basic MapReduce template. We rewrite the template under the new API to enable you to use it in the future."

Exactly two weeks ago today, Hadoop 1.0.1 was released after 6 years of development. Inbetween the version that this book covers, and this most recent version, several intermediary versions were released, which provide bug fixes, improvements, optimizations, and new features, as well as support for some of the offerings in the Hadoop ecosystem. More timely information on open source technologies that enjoy wide community support is always going to be more readily available on the internet, especially via blog posts, but in my opinion this fact does not detract from the value of this text, which still serves as a good introduction to the Hadoop ecosystem, especially for those more comfortable starting out with a published text. Just be aware that you will be quickly referring to other materials after you make your way through this text.

The portions that I especially appreciated about what Lam has to offer include his presentations in chapter 5 on reduce-side joining and creating a Bloom filter, the cookbook that he provides in chapter 7 that includes segments on passing job-specific parameters to tasks, probing for task-specific information, partitioning into multiple output files, inputting from and outputting to a database, and keeping all output in sorted order, as well as chapters 9, 10, 11, which discuss the larger Hadoop ecosystem, especially the introduction to Pig Latin. Recommended to anyone looking for an introduction to the Hapoop ecosystem of technologies who understands that published texts such as this one cannot contain information about the latest releases.

New Book Review: "Confessions of a Public Speaker"

New book review for Confessions of a Public Speaker, by Scott Berkun, O'Reilly, 2011, reposted here:

About a year ago, shortly after one of the directors at the consulting firm with which I am currently employed recommended that I look into speaking on a more formal basis, partially as a way to build my personal brand, I surveyed what was then available in this genre of books and decided to read a book written by Alan Weiss called "Million Dollar Speaking: The Professional's Guide to Building Your Platform" (see my review), which was itself rebranded for the version of the book published last year (2011).

As someone interested in exploring public speaking as a professional, and not as a debater, for which some undergrad professors unsuccessfully tried to recruit me, what Weiss provides, combined with the effort being discussed here by Berkun, is probably all that one needs to read to get a good idea of what one is to expect when trying to succeed in this field of work. While there is overlap between these two books, the core difference is that Weiss offers advice regarding the business side of public speaking, while Berkun concentrates solely on the public speaking experience.

The chapters I especially appreciated in what Berkun has to offer include Chapter 4 ("How to Work a Tough Room"), Chapter 8 ("The Things People Say"), and Chapter 10 ("Confessions"), along with the appendixes entitled "How to Make a Point" and "You Can't Do Worse Than This". Although the last area of the book I list here does not provide explicit advice, it shares 19 stories from a variety of professional speakers that are intended to provide encouragement, and are at times so hilarious that it is worth checking this book out for at least these stories.

In my opinion, this book is so easily accessible because, like Augustine, Berkun is willing to share the good, the bad, and the ugly (at least when it comes down to his public speaking abilities and experiences). While attempting to demonstrate infallibility might help establish credibility, the purpose of this book is to share with the goal of influencing others who practice the same work, rather than to show how the author is so good at what they do to those who might want to purchase his services.

To this point, the lessons learned from a University of Southern California research study in Chapter 8, in the context of receiving audience feedback, were very well placed. After discussing the audience feedback he has received in the past, he points out to the reader that while audience feedback can provide valuable information, many times it really does not say much, and proves his point well by discussing the results of this research study, which demonstrated that even the credibility, superficials, and enthusiasm of an actor in the guise of an expert can skew feedback results.

Recommended to public speakers and anyone interested in public speaking, as well as professionals who are not necessarily looking to be paid for public speaking engagements, but spend some of their time addressing groups of colleagues or clients in their respective professional areas. Informative and entertaining. Well done.

From the Trenches: IBM WebSphere WMB, WTX, MQ

At the moment, I am resurfacing after having spent nearly the last 6 months in the trenches of IBM WebSphere Message Broker (WMB) 6.1.0.9, IBM WebSphere Transformation Extender (WTX) 8.2.0.4, and IBM WebSphere MQ 6.0, a set of products that were inherent constraints in the technology migration project I led at one of my recent clients.

For those who are familiar with IBM WebSphere Application Server (WAS), the flagship in the WebSphere product line, but not familiar with these products, it might be surprising to hear that WAS is just one of hundreds of products for which IBM has chosen to brand "WebSphere".

Many of these products have pre-IBM histories. For example, the first incarnation of WTX was Ascential DataStage TX, and later became DataStage TX and Mercator Integration Broker. Some of this information is readily available, but more than likely one will come across these facts when searching the forums.

Another example of past lives is WMB, which was initially developed by NEON (New Era of Networks), later acquired by Sybase, and initially known later as IBM WebSphere MQ Integrator. Although one will likely come across this information in the forums as well, some processes have retained their "integrator" names, much akin to Orion processes still retained within Oracle Application Server long after the Oracle acquisition, a fact of which I became keenly aware after running across the first exceptions on a past project of mine.

In some very limited respects, I liken the WebSphere product line to the Apache Software Foundation. On another recent project, an individual who had been with my client for quite some time had initially been inquiring about Apache configuration, to which I responded, "Umm, Apache what?" "Oh, Apache HTTP Server". It is very common for professionals to simply refer to "WebSphere" rather than "WAS".

Plenty of resources regarding the purpose of WMB, WTX, and MQ exist out there, so the information you will find here on purpose is brief. However, what I will share is an honest review of what I consider to be some pros and cons of this software suite, based on my experience from the trenches of a recent project.

One aspect that I consider a bit lacking with reviews out there is that those writing the reviews have not actually used the software in any meaningful way. The same goes for the many presentations I have read, which have come in handy to some degree to increase awareness of available product options, but separating the wheat from the chaff can be done rather quickly after some initial exposure is gained.

From my experience for this specific project, when comparing against what can be seen as more manual options, I tend to think that there are benefits to this particular tool suite. To compare these tools to other commercial products, however, is a bit out of scope for this blog post.

Integration with queues and web services is relatively less complex than would otherwise be the case. In addition, I personally think it aids in the development process when processes can be seen visually via the Eclipse-based integrated development environment (IDE).

Speaking of separating the wheat from the chaff, I am cognizant of the fact that using this tooling on what might be considered greenfield development is likely to be quite different than a technology migration, which inherently has significantly more constraints. And quite honestly, there are a host of project factors not related to the actual development of which I need to remind myself are out of scope for this blog post.

Bugs Abound

However, to cut to the chase, let's face it, there are some signficant bugs with this tool suite. Working with these bugs brings to mind the conversations I had a couple years ago with a former colleague of mine who was a Microsoft .NET expert. While I may have led that enterprise architecture project, I was his Java technologies counterpart, and I learned a great deal from him with regard to Microsoft technologies.

One of the many take-aways from my conversations with him was that although Microsoft .NET development staff might be seen as less expensive at a surface level than Java development staff, it is common that Microsoft .NET developers tend to rely on drag-and-drop features of the IDE, and once more knowledge below this surface level is needed due to exceptions or more advanced features, this facade tends to give way to the fact that deeper level expertise is needed.

Handling WMB and WTX bugs was like running a steeplechase race. While it is true I ran a sub-5-minute mile when I was younger, I never ran hurdles due to the injuries my teammates encountered (some of which led to leaving the team), and it pains me to think of what "gifts" the steeplechase could have granted (I still consider running a true gift). Like a steeplechase, however, depite the obstacles and injuries we continued to move forward throughout the duration of the project.

Workspace Corruption

The first bug area was frequent WMB workspace corruption. I have never experienced workspace corruption in all my years using Eclipse. Eclipse users pointed in the right direction as to how to recover from such corruption, but due to the nature of the project, doing as suggested did not really save any time compared to simply creating a new workspace and checking out from source control (in this case, CVS).

The problem with creating a new workspace each time this happened (the frequency of occurence varied considerably, from once every few weeks to multiple times in a single day) is that the project structure was less than optimal, and because this was not greenfield, I was constrained to using the structure despite alternate recommendations to simplify maintainability.

Of course, recognizing when the workspace became corrupted was an art in itself, because it came in many forms. With WMB, one can build deployable broker archive (BAR) files using the IDE or the console via the command "mqsicreatebar". Building via the IDE provides more feedback, but unfortunately it also provides very little flexibility in terms of configuration, so what we ended up doing for each work product was move to the command line once core pieces were successfully built via the IDE.

Command "mqsicreatebar" provides significant benefits over the IDE, but builds are either successful or end in with an error code of 99, with no explanation. The manual that IBM provides offers no help, other than to state the obvious. Along with these error codes, there are also what I came to call sub-codes, so it appears that the compiler attempts to offer some additional feedback, albeit with no reference to interpret.

Workspace corruption tended to end with a sub-code of 13. The only way we figured out how to resolve this error was to restart WMB Toolkit (the name of the IDE). Simply performing project cleans had no positive effect. Early on, before I realized how little feedback I would get upon error, some single-digit sub-codes were also received, but these were most often due to typos in the override properties files that can be provided in conjunction with "mqsicreatebar". The exception was a sub-code of 2, which evidently signaled a successful build, with subsequent successful unit tests, but the tooling complained nonetheless.

Odd Behavior

The second bug area is associated with the odd behavior of WMB Toolkit and WTX Design Studio. Examples abound, but two immediately come to mind. What is interesting about at least a portion of these bugs is that contributors to the developer forums dismiss these as being "well known" by the community, with some that persist over several releases of the product, but never seem to get fixed. Developers using this tool for the first time will long for open source software with strong communities that have a mission to provide continuous improvement.

One example is the annoying behavior of WTX Map nodes in WMB Toolkit. Each WTX Map node points to a compiled map created by WTX Design Studio. If the compiled map represented by the underlying WTX Map node property does not exist, the output "terminal" of the node will disappear, and the outgoing flow originally associated with the previously existing terminal will be flagged. Even worse, the WTX Design Studio perspective of WMB Toolkit and WTX Design Studio itself decides seemingly sporadically when it wants to clean projects, often at inconvenient times.

The best example of this behavior is when a BAR has just been built and deployed in the local development environment. WMB Toolkit provides a convenient initial local unit test environment, but when unit testing commences after this first deployment, execution will fail. At first experience, the reason why this execution first fails is a complete mystery. I remember reviewing all of my work, step-by-step, the first time this happened to me, to determine what had gone awry.

Because of this first experience, and many, many other similar situations since, I have become an expert at reading trace logs, which came in handy because client developers were accustomed to using the graphical debugger alone, a convenient but incomplete debugging solution. Of course, the trace log facilities typically consumed 50 GB to 100 GB on a daily basis, so removing trace files each morning became a daily habit.

After deployment, WTX maps to which each of the associated nodes are pointed need to be recompiled. After this recompilation, the nodes are then able to find the binaries for these maps again, but even though output terminals of nodes reappear, the outgoing flows need to be manually dragged back to these terminals. A developer should not need to do this on so frequent a basis, and when multiple WTX map nodes exist within a given WMB flow, this process is exceedingly ardous, and frankly, a waste of time that would never survive a Lean development process.

Another example of odd behavior appears to be some type of workspace caching issue, and this problem arises in several different forms, with two being the most prevalent. The first is associated with type trees created in WTX Design Studio. In order to map field values from one file definition to another, whether each is an XSD, flat file, COBOL copybook, or one of many other possibilities (one of the strengths of this tooling), a type tree with an MTT file extension needs to be created for each file definition.

However, after the initial creation of an MTT, the type tree can be further tweaked at the properties level in a host of different ways. The problem is that in many scenarios, the modified type tree may not be immediately available in the workspace even after WTX Design Studio signals a successful build. As a new user of this tooling, I was initially inclinded to think that I had done something wrong when availability of a new type tree did not seem to exist. After a lot of trial-and-error, it was determined that a project cleaning, in addition to restarting the IDE, was needed in order to fix this issue.

After a while, we became accustomed to this bug, but a related, more serious bug was when we occasionally saw type trees from other unrelated projects become available from a given project. The fix was the same. Project cleaning is not enough. Restarting WTX Design Studio was also necessary.

Poor Documentation

I extend the definition of "bug" beginning with the third bug area. At first glance, or even after some light usage following the building of initial proof of concepts, documentation for this tool suite appears to be thorough. And in fact, I consider some of the documentation rather good. For example, the "Functions and Expressions" manual for WTX is reliable and provides some good examples for creating data rules.

However, the reliability of some isolated manuals does not rub off on other manuals and documents. Before discussing further, be aware of the fact that when I discuss poor documentation here, I exclude qualms that are related to documents that are somewhat deceptive due to not being available for a specific release of a product, or those that do not at least flag readers to the possibility that a document as currently written may not provide screen shots immediately pertaining to the stated release reference because of enhancements that may have been released along with some mid-release bug fixes.

For example IBM Redbook "WebSphere Message Broker Basics", the book that I first read to ramp up on WMB Toolkit, includes quite a few misrepresentations, even though at first glance it appears to be a complete, step-by-step introductory guide. I still ended up creating some initial proof of concepts based on this particular guide, but it was arduous at times to figure out the actual way to do things in WMB Toolkit that were explained incorrectly.

Examples of how the actual IDEs are not self-documenting also abound. If it were not for the urgency of the enterprise that my team complete the technology migration within the specified time period, I can easily imagine a typical developer not coming across many of these issues until after years of usage, after the tooling has already engrained itself in the organization.

One example is associated with the type tree functionality mentioned earlier in this blog post. Specific to COBOL copybooks, what I consider a prominent bug is related to 88-Levels. For those not familiar with 88-Levels, COBOL provides a mechanism within a copybook file definition that essentially permits a program to refer to the specific value of a variable by using a specified name. Essentially an alias for the value, this mechanism is intended to help prevent hardcoding of commonly used values within the code.

However, WTX Design Studio in my mind has corrupted the purpose of this language mechanism. After unit testing a transaction that contained an 88-Level definition in one of the associated file definitions, I became quickly aware that 88-Levels actually restrict the corresponding data values for the associated variables. When browsing properties for such a type tree, you will notice the "Restrictions" property, which has a nested property called "Rule" that can be one of two values, "Include" or "Exclude".

Since an 88-Level is not intended to be a rule, per se, when confronting a map that fails because of data provided for an 88-Level field, one might be inclined to choose "Exclude". However, this option does not exclude the variable as a rule. What this option does, in the words of the "Type Designer" manual, is "exclude the restriction values as invalid data" when "Exclude" is chosen, and "include the restriction values as valid data" when "Include" is chosen.

When explaining this issue to the client developers, they did not understand the issue. The problem is that 88-Level fields are intended to simply provide convenience and maintainability, and are not intended in any way to somehow restrict the values that the field can hold. The only options that WTX Design Studio offers in this regard are to include or exclude restriction values, but the 88-Level values are not intended to be restriction values.

The only option is to re-create the type tree after removing 88-Level entries from the associated COBOL copybook. However, this brings up another issue. Should it really be considered a best practice to start manipulating legacy file definitions? With greenfield development, removal of 88-Level entries might be considered a valid option, but not if traceability to source systems is desired. This issue remains logged in the JIRA issues database I set up for the project, with no clear path to resolution.

The lack of effective documentation for command line commands such as "mqsicreatebar" was already mentioned earlier in this blog post. While not a rampant issue, there are quite a few cases in the tooling where the developer can be misled down the wrong path until unit testing is actually performed to determine tooling behavior. The best example of this that comes to mind is associated with XML validation.

After creating a map within WTX Design Studio, an option called "map settings" can be accessed either in WTX Design Studio itself, or via a WTX Map node within WMB Toolkit that points to the map. One aspect of map settings to which we continually returned was the level of map tracing that was needed. Map tracing is valuable because it provides detail behind map errors, similar to the flow-level tracing that was mentioned earlier in this blog post.

Map settings also provide other properties that can be customized based on need, and one setting that we eventually used, before incorporating XSLT nodes, permitted the ability to turn off XML validation. However, discovery of this option was not trivial, and was so inappropriately named that I am not able to find this setting as I write this blog post. The apparent ability to turn validation on and off at the SOAP Request node level is also a bit esoteric.

Small Community

Last of all, the developer community surrounding this tool suite can be considered another bug area. As mentioned earlier in this blog post, "developers using this tool for the first time will long for open source software with strong communities that have a mission to provide continuous improvement". In addition, the forums associated with this tool suite seem to be dominated by a handful of unsupporting individuals that routinely request that those seeking assistance refer to the IBM product information centers or Redbooks before attempting to request for help. A commercial product with paying customers should simply be better equipped to help. And because much of the assistance sought concerns matterns that should be easily ascertained from the tooling, it is likely that this tool suite exhibits an unnecessarily high level of complexity.

Personal Reading List: 2Q2012

My personal reading list for April, May, and June 2012.

Color Key: Special Notes, Currently Queued, Completed Reading.

---

Data Mining: Practical Machine Learning Tools and Techniques (Third Edition), by Ian H. Witten, Eibe Frank, and Mark A. Hall, Morgan Kaufmann, 2011. By invitation from Amazon Vine.

Everyday Enterprise-Architecture: Sensemaking, Strategy, Structures and Solutions, by Tom S. Graves, Tetradian Books, 2010.

Gold and Iron: Bismark, Bleichröder, and the Building of the German Empire, by Fritz Stern, Vintage, 1979.

HBase: The Definitive Guide, by Lars George, O'Reilly, 2011.

Introduction to Decision Analysis (Third Edition), by David C. Skinner, Probabilistic Publishing, 2009.

Portlets in Action, by Ashish Sarin, Manning Publications, 2011.

R in Action, by Robert Kabacoff, Manning Publications, 2011.

Scenarios: The Art of Strategic Conversation (Second Edition), by Kees van der Heijden, Wiley, 2005.

Strategy and the Fat Smoker: Doing What's Obvious But Not Easy, by David H. Maister, The Spangle Press, 2008.

Waging War on the Autistic Child: The Arizona 5 and the Legacy of Baron von Münchausen, by Andrew J. Wakefield, Skyhorse Publishing, 2012.

What is a P-Value Anyway? 34 Stories to Help You Actually Understand Statistics, by Andrew J. Vickers, Addison Wesley, 2009.

With the Light: Raising an Autistic Child - Volume 2, by Keiko Tobe, Yen Press, 2007.

The Wrong Answer Faster: The Inside Story of Making the Machine that Trades Trillions, by Michael Goodkin, Wiley, 2012. By invitation from Amazon Vine.