Metrics and Measuring Objectives

 

#4) Assignment 4 – Metrics and Measuring Objectives

Part 1 Problem Statement:  One of the strengths of internet enabled technologies is that they lend themselves to recording information and generating data. As we have seen in the Air France case as well as in the readings metrics can be used to determine operational process, explore corporate financial issues, expand products/services and gain important consumer information. Metrics can be used to make the company smarter, quicker and more responsive to the consumer base. It can, due to the nature of the amount of information available, overwhelm a company with too much or not useful data.

After reading the Air France case refer to the four questions that are asked in the final “Next Steps” section of the case. Think through your answers carefully and provide documentation from the case data to support your answers when appropriate.

Questions: Should Media Contacts recommend a uniform strategy for Air France across search engine publishers? Or would it be more effective to tail each publisher strategy to maximize return on investment?

Answers:

From the article, the optimal search marketing campaign is based on choosing effective allocation of ad dollars across the various search engines, as well as selecting appropriate keywords and bid strategies for placement on the search result page for Internet users. In determining the optimal strategy, the case presents background information on the airline industry as well as the Internet search options available at the time, including Google, Microsoft MSN, Yahoo!, and Kayak. Additionally, background information is provided on SEM and its associated costs and means of measuring the successfulness of each marketing effort. The table Exhibit 6 included line item information for keywords and keyword groups for each campaign and search engine, implying that strategies between each search engine is at least slightly difference, although high level conceptually they all lie on similar principles.

However in another article, the author suggests that buyers using search engines to look for information tend to trust and follow links displayed in the editorial section of the search-results page. Most on-line sellers, however, do not invest in search engine optimization (SEO) to get higher search-results rankings for their listings, but instead prefer paid placements. They explain that SEO is more expensive than paid placements, produces results that do not justify its cost, and does not consistently lead to high search-results rankings. This implies that sellers would invest in SEO if it were less expensive and its rankings were more consistent. However, even if SEO and paid placement cost the same, and SEO always produced high rankings, paid placement would still be the search engine marketing (SEM) strategy of choice for most on-line sellers. Interestingly, no optimal SEM strategy includes SEO. These findings are especially significant for advertising professionals who have to justify investing in paid placements despite overwhelming evidence that buyers ignore them and follow links in the editorial section of the search-results page. (Sen, 2005)

Questions: How can campaigns be improved to increase overall value gained from investment with a search engine publisher? Should keywords be added or dropped from the campaign? Should campaign tactics or copy be adjusted to improve campaign performance?

Answers:

To optimize a sponsored campaign, the campaign had to improve on one or more of the following: cost-per-click reduction, increase in bookings, net revenue, revenue per transaction, return per transaction, overall performance by engine, or other performance metrics that could improve the net revenue of a particular campaign.

We need to understand how to maximize the net revenue and the ROA of its Internet marketing campaigns by evaluating alternative strategies.  We also need constantly look to improve our online performance through innovative thinking and testing new approaches. The level of analysis and reporting we receive from them is crucial to the continuing success of our business. (Jeffery, 2007)

Questions: What are the most important KPIs, and what impact will campaign changes have on these KPIs?

Answers:

Search Engines Bid, Clicks, Clicks Changes, Avg cost per click, impressions, engine click thru%, Avg Pos, Transaction Conversions %, total cost/ transaction, amount, total cost, volume of bookings.

Index	Target
Search Engines Bid	increase
Clicks	increase
Clicks Charges	increase
Avg cost per click	decrease
impressions	Increase (it actually depends if people want to see it or not)
engine click thru%,	increase
Avg Pos	increase
Transaction  Conversions %	increase
total cost/ transaction	decrease
amount	increase
total cost	Delta of cost / delta of amount        - > Decrease
volume of bookings	increase

Questions:  How should future SEM campaigns be structured? In the past, Media Contacts had concentrated on Google, Microsoft and Yahoo; was there now an opportunity to optimize search advertising with metasearch companies such as Kayak?

Answers:

We can first determine the key performance indicators for the project to guide analysis and enable comparison of various SEM campaigns. Cost per click and probability to produce a sale differ among publishers. Then we can use a portfolio application model's quadrant positions to determine optimal publisher strategies. Additionally, pivot tables help illustrate campaigns and strategies that have historically been most successful in meeting Air France's target Internet sales. Multiple recommendations on how Media Contacts can assist Air France in improving its SEM strategy can be derived from the data provided.

Kayak does have its competitive advantage. Kayak is considered a travel aggregator of sorts, meaning it sourced the Internet for travel business and services. As a SEM niche, Kayak also differentiates itself as having an alternative business model, from established travel service aggregators, and as having a unique technology architecture. On the other hand it doesn’t have an impressive historical business results although it has good forecast. I think to determine if it is good or not depends on how likely people are use Kayak as a search engine if they book tickets from Air France.

Part 2 Problem Statement:  For the second part of the assignment consider that you have been asked by a company to discuss their use of metrics. They are seeking advice as to how to design a sensible metrics program as they wish to evaluate how their website is helping their company’s business. They are not really sure of how to design the program and want your advice as to how the firm should proceed. They have heard that a strong argument could be made that metrics are generic across companies. Namely, all companies need to be concerned about customer awareness, positive brand attitudes, low conversion rates, high satisfaction and loyalty. They have also heard that others argue that customer metrics can provide the earliest warning of future troubles and potential areas of needed development. Others, they hear, have argued that the internal metrics in particular, employee metrics (e.g., retention, satisfaction) are better indicators.

In addition, they tell you that they have heard that the specific metrics relevant to a firm might vary by the life stage of a company. That is, early emphasis should be on customer acquisition with concerns about profits in later stages. Others have argued that this approach is not realistic and that every stage of a company's evolution should have a scorecard type of evaluation.

Questions:  How all the metrics fit together and how they are to be used has confused the firm and they are turning to you to help them put these issues into some perspective. In this part of the assignment you are to write a two to three page summation for the firm about what they should consider when reviewing a metrics plan, how to organize the plan and what the outcome of it would be.

Compose your answers in a Microsoft Word document and attach the document using the Add Attachments button. Remember to use the Submit button to complete your submission. Be sure to put your name and section on the top of the document.

Answers:

An Effective Metrics Process Model (Program)

We need to use a good effective metrics process model and program to answer all these questions. (Augustine, Schrodeder Date). The following model is used in US Air Force.

Problem Statement

Measurement for the sake of a number or statistic rarely makes a process better, faster, or cheaper. A poor measurement can hurt a process if incorrect decisions are based on the result of that measurement. People at all levels of organizations continue to take measurements hoping that they will shed light on the best way to provide a product or service. Though fraught with good intentions, these poorly contrived measurements add to the confusion of what should and should not be measured.

We need to use a good effective metrics process model and program to answer all these questions. (Augustine, Schrodeder Date). The following model is used in US Air Force.

Metrics Philosophy

First before we create or review different metrics plan, we have to establish and validate our organization goals. What is important to the organization and its customers?

Although these goals are important to every organization, it can be difficult to focus on defining clear, measurable goals, based on what is important to customers. Senior management can be skeptical about the value of spending time defining such goals. Once these goals were proven to provide useful decision points, senior leaders were willing to define other goals of interest to the organization and ultimately to the customer

Validate Metrics

A plan was created to validate the purpose of each metric. Many of the cost, schedule, and performance metrics were relevant because they directly measured the mission. However, the metrics process to collect and analyze this information required updating and validation. We defined an overall metrics philosophy as an adjunct to the strategic plan and noted that each new metric had to have basic information associated with it, making it useful to the right people at the right time.

For example

Metric Title	Brief Description
Link to Goals/Objectives	Decision(s) based on analysis	Who makes decision(s)
Who collects data	How is data collected	How often is data collected
Who reports data	How and to whom is data reported	How often is data reported
Who analyzes data	How is data to be analyzed   (formulas and factors)
Lowest acceptable values	Highest acceptable numerical   values	Expected values
At what point will you stop   collecting this metric

Review and Organize Metrics Plan

To review the data objectively, we surveyed our data collectors and senior leaders to see if they understood why we collected each metric. At least one set of surveys were used as the control group and distributed to model designers because those who designed the metric should understand why the metric is important in making decisions. Through this survey, we obtained raw data on specific problems and accolades associated with each metric (Data analysis). Changes were made to existing metrics to streamline and standardize collection processes, and a number of metrics were deleted. After the new metrics passed a trial period, senior managers were confident that the new metrics provided information necessary in making decisions.

It is difficult in creating metrics that are easy to understand yet help the right level of management make timely decisions based on fact. Many times, the difficulty is that we continually ask how to measure a process rather than determining what decisions need to be made. 

Putting things together

In starting this project, no one knew exactly why some metrics were collected. These measurements were discarded because they were only one-dimensional, leaving the data analyzer and decision maker with questions as.

The right measurements soon became apparent in the goal-definition stage. Originally, goals were stated solely because they were the easiest to collect. Many metrics originally were turned down because they were not easily placed in a bar or Gantt chart. It soon became apparent that by defining the goal, the metric becomes obvious, rather than defining an easy metric and trying to make a goal based on it.

After much deliberation, the goal became clear, quantifiable, measurable and align with organization strategies. With this clear, measurable goal in mind, metrics were created.

Conclusion

If organizational goals are written clearly and are measurable, creating a metrics program becomes simple. A successful metrics program ensures that data is collected and analyzed consistently, and most important, this program ensures that the right people are making timely decisions based on fact. All that remains is a semiannual review to ensure that you stay on track with the decisions your organization is making based on these metrics.

We should take this analytical view of your metrics, thinking not of individual measurements but of a system that helps you make good organizational or corporate decisions.

Reference

Augustine, Schroeder. (Date). An Effective Metrics Process Model

Mark Jeffery. (2007). Air France Internet Marketing: Optimizing Google, Tahoo!, MSN, and Kayak Sponsored Search

Ravi Sen. (2005). Optimal Search Engine Marketing Strategy; Internal Journal of Electronic Commerce

Eric Tse, Richmond Hill, Toronto
Tse and Tse Consulting -Security, Identity Access Management, Solution Architect, Consulting
http://tsetseconsulting.webs.com/index.html
http://tsetseconsulting.wordpress.com/
http://erictse2.blogspot.com/