In my previous post ( http://ianalderton.com/?p=861) I disused that challenges of extracting value from Big Data using new forms of competitive advantage, such as dark data and data exhaust. In this post I am going to discuss the challenge of governance for Big Data.
Many organisations believe that Big Data is somehow different and is not applicable to the same degree of governance rigour. This couldn’t be further from the truth. Big Data has the same hallmarks as small data when it comes to governance. The key differentiator is the volume, velocity, variety and volatility of information being processed.
Competing On Insight
The search giant Google was one of the pioneers in Big Data, as seen with the launch of their BigQuery analytics tool, an area that has now developed into an industry worth billions of dollars. Big Data provides the opportunity to develop new operating models and new products based on real time customisation and pricing.
Data As A Currency
With the new analytical tools comes the ability to aggregate data. Organisations can now combine an internal view of their customer with geo–spatial information, such as Foursquare and Twitter, to generate targeted offers and promotions. Without a doubt, data is a new currency that can be traded and invested to drive customer engagement and product differentiation.
With this explosive growth of data and real time analytics comes greater responsibility to balance the need for knowledge with the right of the individual. Data by its very nature is contextual. When an individual shares information on Facebook, they are sharing stories, news, and information with their trusted network. Big Data can identify patterns with unintended consequences when aggregating data.
A recent and prominent example, from the US superstore Target, was when a man discovered his teenage daughter was pregnant because coupons for baby food and clothing were arriving at his address from the store. The daughter, who had not told her father she was pregnant, had been identified by a data system that looked for pregnancy patterns in purchase behaviour.
With Big Data we are entering a new era of ethical convention and challenge. The relationship between the social citizen, the customer and the organisation is changing. Big Data can pick up on trends and insight with unintended consequences.
Governance
With Big Data comes greater opportunity to increase customer engagement and competitive advantage. At the same time this represents a significant moral and ethical hazard for the uninitiated. Organisations need to be able to compete with greater responsibility and balance the need of the individual and customer versus the right to access data as a new currency.
Privacy
In 2006, AOL released anonymous search results in to the public domain. Unknown to AOL, this information was re-engineered by an individual who was able to make it no longer anonymous, thereby identifying the original search details. This information was then aggregated by the New York Police department, across their own photo records and face recognition software, to arrest an individual for attempted murder. In this instance, the New York Police department got their man but with these new tools, comes a new ethical dilema. There is a moral responsibility to ensure that there is clear definition of how the data will be used and potentially aggregated.
The customer’s expectations of trust has to be managed alongside privacy laws and regulation. When the data is collected, if the customer agrees for his information to be used only internally to the organisation, aggregating the internal view of the customer with his facebook profile is clearly a violation of privacy rights, as was illustrated in the Target example above.The sensitive nature of information is critical for all organisations.
In my experience, that are 7 key steps than need to be managed for effective governance of Big Data :
1. Define the business stakeholders and business benefits for Big Data
Make Big Data a business driven initiative. Define who is responsible for owning and maintaining the data
2. Define the organisational structure, scope and governance model for Big Data
How much and what type of data can the organisation consume from the clients public profile? Has the customer agreed to the aggregation of their internal and external profiles ?
3. Define Big Data custodians for both internal and external data
Who owns the internal and external view of the customer within the organisation ?
4. Definite risk factors and controls for Bug Data governance
What are the new laws, regulations & privacy standards for data aggregation?
5. Document policy and procedures for Big Data
How is the data maintained and audited to ensure integrity and compliance?
6. Define quality management policies
How is the data managed and maintained for accuracy and integrity?
7. Information Security and Privacy
How does Big Data change the concept of information as a corporate asset? How do all these Big Data technologies relate to our current IT infrastructure?
Only by implementing a clear and structured data charter, with an increased level of rigour and due diligence, can an organisation be confident that it has the correct level of governance.
To Conclude
Big Data brings a new era of moral hazard and commercial ethics. Against a backdrop of phenomenal scale and power to uncover commercial insights, Big Data can generate unintended consequences without the correct level of stewardship and governance.
Organisations need to revise customer interaction and data charters on how they aggregate the internal view of the customer with information available in social and public domains.
As Big Data becomes increasingly important, and new disparate data sets are shared and aggregated, the stewardship of how this information is consumed and aggregated will be central to both data privacy and how an organisation maintains its trust and integrity with its customers.
IAN ALDERTON
Email : ian@IanAlderton.com
Tel : +44 (0) 7702 777770
