Matching v. Searching
A blog on matching, and why it's better than searching, with a slight bias toward iXmatch, by Bret A. BusseTechnorati Profile Blogroll This
7.30.2003
Sometimes you don't know what you don't know
Another great customer quote. Like most people, you probably never get past the first or second page of search results. Think about all the stuff you never even see. There might be a whole set of products or articles that would be interesting to you, but you don't even know they're there.
Text clustering and dynamic differentiation give you a clear picture of everything available. You can select interesting clusters or features and see what's out there.
7.29.2003
Don't put your buyers in a box
As I mentioned last week, our customers and prospects often come up with great ways to describe what we do.
After showing iXnavigator, our dynamic navigation product, to an auto manufacturer, he said "This is exactly what I've been saying we need to do. Right now, we put our buyers in a box by making them select a make and model first. You guys don't put them in a box because they can tell us what's important, and then we can tell them which makes and models best fit their needs."
We couldn't agree more.
Matching is bi-directional
Search only goes one way: you ask for things that contain your keywords and get a list. Matching allows both sides to participate. In every situation, each side has something it requires and something it offers.
In HR, a candidate offers her skills, abilities, experience, etc. and requires a salary, benefits, perks, etc. Bi-directional matches mean the candidate fits the job, and the job fits the candidate.
In dating, each person has certain attributes and interests, and other things they're looking for. Bi-directional matches mean both people "fit".
I'm sure you can come up with plenty of other examples.
You simply can't satisfy both sides' requirements with plain-old search.
7.25.2003
Guided stumbling
Customers and prospects are often far better than us at describing what we do. After showing a guy a demo of our software, he said, "That's so cool. It's like guided stumbling." I laughed, and then realized he's exactly right.
When you begin searching for something, you don't know what's there. Other software products either give you a huge list of results, with no guidance, or a predefined path to follow, which doesn't allow you to stumble. If an interesting product or article isn't on the path you select, then you never even have the option of stumbling across it.
Because we make every product and every article available, and give you every possible path, with a dynamically-generated view of all of the options, we really can gracefully guide you as you stumble toward exactly what you're looking for.
7.24.2003
CombineNet
One of the great things about my role here at iXmatch is the opportunity to meet some really great, and really smart, people. Yesterday I had the pleasure of meeting Dr. Tuomas Sandholm, CTO, and Tony Bonidy, CEO, of CombineNet. These guys specialize in software to help large corporations optimize sourcing. This involves evaluating multiple bids and multiple criteria, and identifying the best overall offer. To me, this seemed like very specialized matching. Turns out their approach is very different from ours (you'll have to talk to Tuomas, or our scientists to understand the difference) and is extremely effective for their customers.
It's always great to talk to someone who completely gets what we're doing, and learn how others are solving complex problems related to matching. I'm sure we'll hear more great things about them.
Business intelligence
Matching customers to offers and offers to customers can also be a complex problem. Determining which offer a customer is most likely to accept, or which customers are most likely to accept an offer isn't enough. Determining why will drive sales.
If you're only using simple analytics to recommend products, then you're probably just saying people who have purchased the same products as you have also bought these - therefore, you should like them. There's no way to determine what it was about the products that lead to their purchase.
Our approach to analytics (and everything else for that matter) is to look at those products as bundles of features. That allows you to say people who have purchased products with features similar to the features of the products you have purchased, have also purchased products with these features. Sounds complex - and technologically speaking, it is - but in a marketing campaign, it allows you to promote the features of the recommended products, which is the why.
Here's an example: Radisson Seven Seas Cruises uses our software to determine to whom to market each upcoming cruise, and then promote the why. The data they have on you and me might suggest we are both good candidates for the cruise to Tahiti in December. If we both receive a regular old offer, it might look ok. But, if you receive an offer that highlights the three primary features that lead to the recommendation for you, say that it's on the Navigator, their best ship, that it's December, and you can go scuba diving, and mine highlights that it's only seven days, there will be activities for my kids, and there is a wine-tasting class onboard, we're both more likely to get excited about it.
7.23.2003
User experience
Matching complex profiles of jobs and candidates requires quality data. Getting it from candidates has to be simple. People like John Sumser and Phil Wolff, of Candidate Voice are extremely interested in "improving the value and quality of the job hunter's experience." They know candidates prefer to use their time effectively. We coldn't agree more.
We pitched our software to Microsoft. Here is our take on how they can create a great a candidate experience and help recruiters sort through the hundreds of resumes they receive for each job posting using our matching software.
Human capital management
Matching people to jobs is a natural fit for our software. When you think about it, people and jobs are extremely complex. There are billions of combinations of skills, abilities, experiences, and interests, all of which can have different priorities for each person and each job. Matching the two together is one of the most difficult problems to solve.
Like my home-buying analogy, finding the best-fit candidate or job inevitably involves making compromises. If you fully describe the perfect job or employee, it's unlikely they exist. Seeing how close you can get is where matching comes in.
Knowledge management
For knowledge management, we match articles to a person's interests.
Pinpoint, from Sagebrush, uses our metasearch engine to send a query to a school district's libraries, subscription databases, selected Web sites and the Net via Google. It merges all the results together, and clusters them based on their similarity.
Beyond that, our software learns the relationship between the person's grade level (grade school, junior high, high school, teacher, parent, etc.), the subject being researched, and the sources selected, so it can recommend and optimize the metasearch parameters and rank the results appropriately.
7.22.2003
Transportation optimization
For supply chain, matcing boxes to truckloads is a perfect application of our matching software.
Transportation Advantage, from HighJump Software, uses our optimization engine to take all of the boxes to be shipped, all of the trucks available, all of the destination points, mileage costs, stopping costs and delivery windows, and matches them all together to come up with the optimal mix of trucks, boxes for each truck, and route for each truck. It will even tell you if you're better off sending a few of your boxes via UPS. It does all of this in about one minute, compared to several hours for Manugistics or i2.
Matching is ubiquitous
At least we think so. As odd as it might sound, product catalog, job board, personals, knowledge management, business intelligence and supply chain applicaitons all look the same to us. The supply and demand sides of the equations in all of these applications have collections of features and attributes, all of which can be matched, scored and ranked. Bet you didn't think a book in library catalog and a box on a truck had so much in common, did you?
7.18.2003
7.17.2003
"query discovery" software
Sleuthing Out Data, by Fred Hapgood, takes a look at "making it easier for users to explore huge volumes of data". He includes an example of how Endeca is being used by Arrow Electronics to help design engineers find products. "Unlike line managers, designers seldom know what they are looking for ahead of time. They start with a wish list of properties for the perfect part, filter out candidates that come close but not close enough, and then find the best compromise by carefully comparing the remaining parts and fine-tuning their design. The very last thing they learn is the part number. Customers such as those require a very different set of searching tools."
This is a great point, and Endeca's approach works for products with a short list of available features. If however, there are lots of features available, then there is still the problem of either showing too many features at a time, or relying on someone to decide which features to present to all of the customers. The hope there, of course, is all of your customers are interested in the same few features. Our approach of dynamically differentiating the products, and personalizing the list of features, is far more effective for these kinds of products.
7.16.2003
Matching is like buying a house
Home buying inevitably involves making compromises. I want a short commute, but I also want a big yard. I need at least a two car garage, but three would be great. I need at least four bedrooms and two bathrooms, and a wood-burning fireplace would be nice. I'll take a little more of this, and a little less of that, as long as it fits my budget. If I describe my ideal house, it probably doesn't exist. Either it's too far away, or too small, or costs too much, or ...
Regular search doesn't solve this problem: It simply filters houses based on what I select. There's no way to say "a little more of this and a little less of that". Matching lets me describe my ideal house, and then scores all of the available houses to see how close I can get.
Catalog navigation
In this article, John Staubly says the key to helping customers navigate through catalogs and to buy stuff is to "Create Well-Defined Buy Paths" because "Typically, consumers have an idea of which product they're interested in and the path they'd like to take to purchase it."
I disgree. In an earlier post, I made the point that if your customers don’t think about your products in exactly the same way as you do, they’ll never find them. Our approach is to collapse the hierarchy and identify which features make the products different. By selecting these differentiating features, a customer can quickly zero in on the products that have the features that actually matter to her – regardless of where the products currently reside in the catalog.
SQL v. matching
SQL is really just a filter. You use it to say "Show me exactly what I'm asking for." The assumptions are you know exactly what to ask for, and whatever you're asking for is actually in the data you're searching through. If those two assumptions are true, that's great, and SQL works just fine.
But if they're false, then the trick is to get some help deciding what to ask for, and seeing how close you can get. That's where matching comes in.
7.15.2003
Query Disambiguation
In his article on query disambiguation, Arnaud Fischer says "Search engines should be more proactive, learn for the benefit of individual users and become smarter over time." He also says "Emerging concept-based clustering technologies, used in search engines such as Vivisimo, are doing wonders at allowing users to refine ambiguous queries."
Unfortunately, Vivisimo does not do text clustering. They use a technique called indexing, which simply displays a list of the words that show up most frequently in the results.
iXmetafind does text clustering and gets smarter over time, as described in this article.
Bundles of features
An interesting way to think of a product is as a bundle of features. If a customer comes to you with a need, you should be able to offer him a product with the features to address that need. The product that has the best collection of features for him is really the one he wants – but odds are you’re forcing him to pick the product first, and then showing him if it has the features he’s looking for.
Here’s an example: I need to stick some plastic to some metal. And it’ll be in sub-zero temperatures for over an hour. And I need to be able to remove the adhesive easily when I’m done. Now, what do you have that will work for me? I’m expecting to see some sort of glue, but it turns out there are a few different kinds of tape that will work even better for me. If I had to pick a product, or even a category, first, I never would have known.
The reality is that your customers don’t really care what your product is called, or what product line its in, or what division of your company makes it. They just have a bunch of requirements and preferences. You have bundles of features called products. Matching the two is what creates a sale.
“Architecture has outlived its usefulness.” Train of Thoughts, John Lenker.
Most product catalogs rely on a combination of keyword search and a fixed hierarchy. I’ve already addressed the problems with search. But the hierarchy thing is equally problematic because fixed information can’t possibly work for everyone.
You might think you know your customers inside and out. But if your customers don’t think about your products in exactly the same way as you do, they’ll never find them. So what if you collapse your hierarchy and make all of your products available at all times? The trick then becomes helping a customer quickly and easily narrow the complete set to a more manageable number.
One technique to address this is to identify which features make the products different. By selecting these differentiating features, a customer can quickly zero in on the products that have the features that actually matter to her – regardless of where the products currently reside in the catalog. You will be surprised to see which features are most important to your customers when they’re given the chance to select them on their own.
You say potato …
Forrester says, “40% of search failures come from customers and firms using different terms.” People can't be expected to know what your company calls things. Another problem is figuring out how to ask for exactly what you’re looking for. If your search is too broad, you get too many results. If you’re too specific, you get no results. Either way, knowing how to refine your query isn’t that easy. If you add words to your search, how do you know which words to add? If you remove words, how do you know which ones to remove?
One way to improve search is to use a text-mining technique called text clustering. It puts your search results into context by dynamically clustering search results into groups of similar documents. You can then easily navigate through the results by selecting relevant content groups. For example, a search for “mercury” returns thousands of results that are clustered into sub-groups based on context, such as the planet, the element, the Greek god, and others. By choosing the planet cluster, other results are removed from view, and you’re left with results primarily related to the planet Mercury.
7.14.2003
So what’s the big deal?
Let’s say your site gets 50,000 visitors a month. 48% of those visitors search for a product (the initial 16% who search, plus half of the remaining 84%). If 85% of those searches fail, then over 20,000 potential buyers will leave without a chance to even look at what they want to buy. Even if only 10% of those potential buyers were ready to make a purchase, you can multiply your average sale price by 2,000 and see how much money you’re losing every month.
Another consideration: how much money is your company paying its employees to waste time? An InfoWorld report last year showed that an enterprise with 1,000 professional employees wastes $48,000 each week, or $2.5 million each year, due to an inability to locate and retrieve information. It’s hard to imagine a case where that doesn’t impact a company’s bottom line.
Looks aren’t everything
Lots of time and money are spent building nice catalogs with elaborate categories and pretty pictures. Content and catalog management systems focus primarily on storing and publishing content. They make it very easy to design document templates, build hierarchies, edit and store documents, etc. — but they don’t focus on helping people find that content. It seems like the search capabilities should be at least as important as pretty pictures, but they’re not.
Instead, search is simply an afterthought: “We just bought the first search tool we came across.” “We know search is important, but new functions get higher priority.” These people are ignoring the fact that 80 to 90% of people who go online to buy products are searchers. Media Metrix statistics show that “search is the first thing 16% of visitors do.” Jupiter finds that “50% of visitors whose navigation attempts fail turn to search.”
Search sucks
You know it’s true. Forrester agrees: “At 68% of sites, fewer than half of the returned results have anything to do with the actual query.” Media Metrix says, “85% of searches do not return what the customer is seeking.” I’m sure you have your own stories. I know I do. So why do we put up with these inadequate search mechanisms? Because it’s our only option. But is it really?
