Bewitched

"We all make mistakes. Of course, when we make mistakes they call it evil. When God makes mistakes, they call it... nature. So whaddya think? Women... a mistake... or DID HE DO IT TO US ON PURPOSE?"

- Witches Of Eastwick

Ideospheric Introspection

Recently I've used graphs from Google Trends and Nielson's Blogpulse to bolster or refute trends which I've detected with MemeMiner.

Each tool occupies a unique portion of the Ideosphere and a confirmation from all three is probably widespread trend. This "set theory" diagram demonstrates their interaction. Google Trends is the set of Google's user queries while Blogpulse is a set of blog entries. These sets might have an intersection while Dejanews is completely independent of both.

Sub-bandwidths of the Ideosphere can be categorized into five types -

An Ideospheric bandwidth (or "channel") either represents a certain meme (and its sub-memes) or it does not. If it represents a meme, it can denote either a positive or negative connotation. Unallocated bandwidth has three potential states -

a) bandwidth which was never exposed to the current meme

b) bandwidth which actively avoids the current meme

c) bandwidth which accepted the current meme and discarded it in favor of a more desirable meme.

Positive and negative connotations are detectable by associated keywords (i.e. a semantic map).

The cause of unallocated bandwidth is trickier to determine, particularly for memes are which actively avoided. Today's case-in-point is "deficit". In spite of massive increases in consumer credit, the Federal deficit and trade deficit, the general public avoids talk and thought of "deficits" -

The absence of information is often meaningful... if it can be detected.

Zune Meme Rerun

Previous Zune meme analysis

Updated Methodology

Microsoft's iPod competitor, Zune, is still not picking up mindshare or buzz traction. Note the similarity of the "Zune" meme's initial spike and falloff to the Breitbart meme (possible artificial manipulation of Internet propagation). Memes travel through the Ideosphere with finite velocity, it can be difficult to increase meme velocity or penetration with marketing hype.

Air America Meme - Another Successful Prediction

I predicted Air America's demise over one year ago based on this graph, trend is confirmed by Google Trends -

Methodology

SOA Methodology, Part 1

Why do you want Service-Oriented Architecture?

To reduce cost (transaction costs).
To increase reliability.

Most companies, particularly the larger ones, have "organic" information systems which were grown over time, not designed. These systems embody a history of decisions which were appropriate for the company at a certain time and size. But many of these decisions conflict with each other or with future direction. A high-level view of the IT structure shows unnecessary transaction boundaries that create latency, data redundancy, duplication and excessive complexity -

Most companies run past the "economy of scale" of their infrastructure. Real costs are always higher than the theoretical cost of a designed system. As this "economy of scale" mismatch increases, costs increase too fast and redesign becomes more attractive. The Service-Oriented Architecture is today's redesign paradigm. In theory, it should create an integrated and seamless IT nervous system -

That's the theory. The reality of a redesign is different, but still a significant improvement over the legacy system -

A primary consideration in infrastructure design is to minimize complexity whenever possible, in spite of political or legacy issues. The "SOA Methodology" aims to improve project success by minimizing complexity.

Part 2

SOA Methodology, Part 2

Part 1

IBM's strategic SOA design documents apply standard project methodologies (top-down, bottom-up, meet-in-the-middle) to SOA systems, but the result is a confusing mixture of methodology without clear guidelines.

A possible failure of SOA could be the translation of strategic vision into the tactical creation and grouping of code-level objects, i.e. the operational project level.

These "SOA methodology" guidelines make the the following assumptions -

Assumptions

1) Most SOA projects are for large organizations that operate outside the "economy of scale" of their IT infrastructure, resulting in inconsistent information integrity, poor information latency, and marginal IT costs which rise faster than organizational growth.

2) SOA adoption is driven by the need to reduce system complexity.

3) Upper management is committed to function over form.

4) System complexity is inversely related to project success. As complexity increases, the project success rate falls.

Our mission: to reduce project risk by reducing system complexity.

The strategy is to decompose the high-level abstract design patterns of SOA to develop concrete guidelines for project management.

Part 3

SOA Methodology, Part 3

Part 2

The high-level SOA design pattern is three interrelated design patterns -

The "Adapter pattern" appears to be lowest risk. For legacy code, the adapter segment probably has risk and complexity which increases at a linear rate because of the low interdependencies of the connectors.

The "Business Directory (Abstract Factory)" consists of interfaces and routing for new custom code or commercial frameworks. Complexity and risk are higher than for the "adapter" segement, but still within the known domain of normal projects. At an abstract level, delegate this pattern to the Mythical Man-Month Methodology. (for further resolution vis a vis empirical determination of impedance-matched frameworks and team size)

Our greatest area of risk, then, is the "Enterprise Service Bus (Mediator pattern)". It is in greatest need of risk management and reduction of complexity. These complexity guidelines are designed for rock-bottom risk.

Part 4

SOA Methodology, Part 4

Part 3

Follow the ESB Guidelines with a thin-thread prototype model -

1) Evaluate and choose a commercial ESB.
1) Implement the ESB and a security foundation.
2) Create the Business Directory and some security interfaces.
3) Add one or two adapters to demonstrate the thin-thread prototype via security enforcement.

3) Determine interfaces for an application framework or custom-code project.
5) Install and modify the new application.
6) Expand the Business Directory to reference new interfaces.
7) Add adapters to support each new application.
8) Iterate through steps 3 through 7 for all applications.

* We need to address management of WS-Profile here somewhere, as well as integrity of interfaces across adapters and within the Business Directory.

* We need to expand this area to handle business logic.

Part 5

SOA Methodology, Part 5

Part 4

Let's assume that the complexity of an SOA project is driven by its components. A project might include legacy support, commercial frameworks and custom code, but it's worthwhile to characterize the project's primary disposition to assess complexity. We can define a project complexity axis, ranging from lowest (linear) to highest (exponential), which uses Big O notation project complexity as a rough proxy for risk.

Legacy code migrations are lowest risk. The project's main tasks are wrapping existing code in standardized interfaces, data transformation and relatively minor application changes. Complexity may increase at a linear rate relative to functions and codebase size.

Projects with new, hand-written code are highest risk because they include SOA adoption risk and normal project risk. These projects appear at the opposite end of the "Big O" scale, arbitrarily labelled as "exponential" risk. Remember that object complexity tends to be multiplicative.

We've identified the ESB as the area of highest risk. Adherence to the proposed ESB guidelines will tend to reduce the inherent project risk. Ignoring the ESB guidelines will shift the project towards higher risk.

Mythical Man-Month Methodology

SOA Methodology, Part 6

(Still under construction)

Part 5

Mythical Man-Month Methodology Table of Contents

1) IT Feedback Loop - IT systems model the real world but they eventually affect it, too. An SOA-based restructuring should increase the scalability of an organization by reducing marginal IT costs. We need a holistic approach which aligns the SOA infrastructure with the organizational structure.

Yikes.

I haven't seen an IBM SOA document that addresses this yet. But if the IT structure models the organization, then alterations to the organization impact the effectiveness of the model. And changes to the SOA model will alter how the organization functions. The most effective projects will make concurrent, synergistic changes to IT systems and organizational structure..

We need a holistic approach which includes structural changes to the organization.

2) What Is Necessary In Terms of Organizational Complexity

Over time, IT systems become more complex as an organization grows. A particular team size and toolset for successful projects will become inadequate for this new level of complexity. A larger team could handle more complexity but size is eventually limited by the Mythical Man-Month, i.e. inter-team communication costs grow too fast and consume too much time.

Over the past few years, several high-profile outsourced projects have failed, often because they attempted to increase complexity through the brute-force method of adding more bodies to the project. Ideally, we'd like to scale up project complexity through an optimum team size and a toolset which magnifies abilities through automation and generation.

We want an impedance-matched project where success rate and project complexity are driven by team size & toolset. But team size is limited by the concept of the Mythical Man-Month (Fred Brookes), optimum team size should minimize internal communications by concentrating skill and domain knowledge in relatively few members. The main difference between Team A and Team B is their toolset. Sophisticated tools magnify each team member's productivity. In today's world, Team B is (hopefully) using standardized frameworks and high-level design tools coupled to automatic two-way code generation.

3) What Is Possible In Terms of Project Methodology

4) Mythical Man-Month Methodology (m-cubed): 3-Point Strategy

a) Constrain complexity at lowest layer (DB Schema)
b) Constrain complexity at highest layer (GUI)
c) Retain an impedance-matched team size & toolset throughout the development lifecycle -
   i) Re-structure projects around the optimum team size & toolset
   ii) Use application frameworks and commercial applications whenever possible

Wiccan Meme

Long-term downtrend for "wiccan".
What is the deal with you Wiccan chicks?
Dang, one of you must read this blog at least once per day.
Am I on a "watchlist"?

I'm surprised you dare to read, aren't you afraid I'll use the Weirding Way on you?

I still think I love D.
It sounds stupid, even to me.
Why do I feel this way?

I wish I could get back to a job that soaked up my time and sexual energy, this is the suckiest my work life has ever been.

--

What would happen if we kissed?
Would your tongue slip past my lips?
Would you run away?
Would I stay?

Or would I melt into you?
Mouth to mouth
Lust to lust
Spontaneously...
Combust.

Skeptical In San Diego

Email from a reader -

--
"Dear Meme Miner,

You published a graph which shows declining interest in "goth" subculture. This is a deceptive histograph. Please fix it.

P.S. Could you please, please, please post a picture of yourself? Thanks!

Signed Skeptical In San Diego"
--

Well, SISD, here's the original memegraph along with the same query against the Google Trends tool, and two pictures of me!

Time Context

A fictitious example of deduction through time context -

--

12:34 pm - #216.127.72.7 reads Burn a Wiccan entry on my site

12:37 pm - Annoyed Wiccan Witch posts angry diatribe in her journal.

--

Aug 27th - #216.127.72.7 makes last login.
Aug 27th - Annoyed Wiccan Witch leaves for Burning Man.

Sept 3rd - Annoyed Wiccan Witch returns from Burning Man and posts entry.
Sept 3rd - #216.127.72.7 makes long-awaited appearance one hour later.

--

I have no doubt that my deductions are often wrong, but this fictitious one could have a strong correlation between its login times and livejournal entry times.

Sigh.

And then there's the context-based information, so hard to deduce, even harder to prove. But I have to admit, the Annoyed Wiccan Witch does show a sense of morality.

Mind Reading

Off to play pool.
Off to Scottsdale next week.
And off my rocker for goofing around with crazy women.

--

I'm the #19 Google entry for "SOA methodology"!

New hit count high!

Valerie

On Wednesday night I taught Valerie how to play pool. I'd rather be back doing real work, but somehow that doesn't seem possible anymore without a lot of strange games, fake resumes and hidden relationships. I'm not a billiards master, far from it, but I know basic rules and strategy (at least more than Valerie did! ho ho!)

She was cute, probably in her early forties, looked a bit like Michelle Pfieffer with the body of a twenty-year-old. The oddest bit of the night was when she bought ME a drink and hugged on me for awhile.

I'm not that cute and huggable.
I should probably post a new photograph to prove it.