Monday, January 28, 2013
Wednesday, September 12, 2012
Get more value from Big Data technologies – Use it for Small Data Analytics
[ Note : I have published this blog originally at L&T Infotech blogsite - www.lntinfotechblogs.com/Lists/Posts/Post.aspx?ID=38 ].
Big data is often defined by three Vs. – Volume, Variety and Velocity. While this definition captures the essence of Big data, it is limiting when used to define technologies that support Big data. These technologies can do much more than just handling ‘Big data.’ In fact, most enterprises can derive more value by using these for ‘small data’ analytics.
Besides handling large variety of data, these technologies provide new analytical capabilities, including natural language processing, pattern recognition, machine learning and much more. You can use these capabilities effectively for small (or ‘not so large’ data) in non-traditional ways and get more value out of this data.
Here is how –
1. Create ‘Data labs’ rather than just a data warehouse
Big data technologies provide advanced analytical environment. The focus is on analyzing the data, rather than structuring and storing the data. Such environment gives a perfect sandbox for experts to ‘experiment’ with data and derive intelligence out of it. For example, Insurance actuaries can derive specific patterns out of claims history data by linking external factors with loss events and define rules for pricing and loss predictions.
2. Don’t just predict, but adapt continuously to changing realities
Big data technologies provide machine learning capabilities that allow calibrating predictive models continuously by comparing actual outcomes with predictions.
3. Change ‘Forecasting’ to ‘Now-casting’
Big data technologies can help in analyzing large stream of data at real-time, without hampering performance. This capability can be used effectively to provide ‘real-time’ analytics. For example, Insurers can define new products that charge premiums based on real-time risk data emitted by sensors or telematics instruments, rather than traditional approach of calculating premiums based on forecasting of risks.
4. Don’t get constrained by a Data model
Have you ever undergone the pain of living with a data model that no longer supports business requirements? Well, don’t worry anymore. Most Big data technologies support ‘Open format’ and dynamic changes to data records to suit analytical needs.
5. Forget Massive data movements
In big data platforms, the data is co-located with analytical processing involving minimal data movements. Forget about those large, multi-year ETL programs.
6. Save cost with low-cost commodity hardware
Large data warehousing and MDM programs often require expensive enterprise hardware and licensing to support desired level of performance. This expense can be as large as 50% of your total cost of ownership (TCO). The big data platforms are designed to work with low-cost commodity hardware (including bursting on cloud), and most are open-sourced. This can help you slash the hardware/licensing costs significantly.
So the moral of the story is – Big data technologies provide many capabilities that make them an attractive choice for ‘small data’ analytics as well. Be innovative in leveraging these capabilities to complement your current analytics world.
Big data is often defined by three Vs. – Volume, Variety and Velocity. While this definition captures the essence of Big data, it is limiting when used to define technologies that support Big data. These technologies can do much more than just handling ‘Big data.’ In fact, most enterprises can derive more value by using these for ‘small data’ analytics.
Besides handling large variety of data, these technologies provide new analytical capabilities, including natural language processing, pattern recognition, machine learning and much more. You can use these capabilities effectively for small (or ‘not so large’ data) in non-traditional ways and get more value out of this data.
Here is how –
1. Create ‘Data labs’ rather than just a data warehouse
Big data technologies provide advanced analytical environment. The focus is on analyzing the data, rather than structuring and storing the data. Such environment gives a perfect sandbox for experts to ‘experiment’ with data and derive intelligence out of it. For example, Insurance actuaries can derive specific patterns out of claims history data by linking external factors with loss events and define rules for pricing and loss predictions.
2. Don’t just predict, but adapt continuously to changing realities
Big data technologies provide machine learning capabilities that allow calibrating predictive models continuously by comparing actual outcomes with predictions.
3. Change ‘Forecasting’ to ‘Now-casting’
Big data technologies can help in analyzing large stream of data at real-time, without hampering performance. This capability can be used effectively to provide ‘real-time’ analytics. For example, Insurers can define new products that charge premiums based on real-time risk data emitted by sensors or telematics instruments, rather than traditional approach of calculating premiums based on forecasting of risks.
4. Don’t get constrained by a Data model
Have you ever undergone the pain of living with a data model that no longer supports business requirements? Well, don’t worry anymore. Most Big data technologies support ‘Open format’ and dynamic changes to data records to suit analytical needs.
5. Forget Massive data movements
In big data platforms, the data is co-located with analytical processing involving minimal data movements. Forget about those large, multi-year ETL programs.
6. Save cost with low-cost commodity hardware
Large data warehousing and MDM programs often require expensive enterprise hardware and licensing to support desired level of performance. This expense can be as large as 50% of your total cost of ownership (TCO). The big data platforms are designed to work with low-cost commodity hardware (including bursting on cloud), and most are open-sourced. This can help you slash the hardware/licensing costs significantly.
So the moral of the story is – Big data technologies provide many capabilities that make them an attractive choice for ‘small data’ analytics as well. Be innovative in leveraging these capabilities to complement your current analytics world.
Friday, January 20, 2012
Big Data - A solution in search of a problem !
What does Big Data solve in Insurance, that cannot be really solved by traditional technologies? This one seemingly simple question generates a good deal of brainstorming. Let’s keep Health insurance aside (that’s easy) and think about P&C and Life insurance space.
Where is the big data ?
Number Uno is Social data, ever growing and less contextual, but BIG it is.
Then we have policy data over years. We, of course, have a loss history of several years.
We have external risk data sources.
Few companies may also have real-time data streams from Cars (PAYD or commercial fleets), factories etc..
What can we do with Big Data technologies?
Sure we have many problems.
First, we need to know customers better.
How many times we tell them that you can save $400 by switching and then when they ask for a quote, we provide a quote more than their current outgo.
Do we congratulate them when they have a new baby arrival at home?
Do we know that they are looking to buy a car?
With big data, we will be able to co-relate all the seemingly unrelated data sources and link them to derive the actionable intelligence.
Another application is Fraud detection. Some people are out of bars, just because we cannot practically spend time and energy to figure out their fraud. Big Data technology makes is possible and simpler.
What about Risk Analysis? Sure! More the data you have, more you know about your customers, you are likely to predict the risk better.
The real advantage
To some extent, we are doing all this with current traditional technologies as well. More the data, Merrier it is, so big data technologies will definitely help, but is that all?
In my opinion, the real advantage of Big Data is to find the problem (or opportunity) that you do not even know about. When a data scientist dives deep into data and finds patterns and co-relates, there will be an Eureka moment, that will provide you the real ‘intelligence’ hidden in this data. Indeed, Big Data is a solution in search of a problem!
Where is the big data ?
Number Uno is Social data, ever growing and less contextual, but BIG it is.
Then we have policy data over years. We, of course, have a loss history of several years.
We have external risk data sources.
Few companies may also have real-time data streams from Cars (PAYD or commercial fleets), factories etc..
What can we do with Big Data technologies?
Sure we have many problems.
First, we need to know customers better.
How many times we tell them that you can save $400 by switching and then when they ask for a quote, we provide a quote more than their current outgo.
Do we congratulate them when they have a new baby arrival at home?
Do we know that they are looking to buy a car?
With big data, we will be able to co-relate all the seemingly unrelated data sources and link them to derive the actionable intelligence.
Another application is Fraud detection. Some people are out of bars, just because we cannot practically spend time and energy to figure out their fraud. Big Data technology makes is possible and simpler.
What about Risk Analysis? Sure! More the data you have, more you know about your customers, you are likely to predict the risk better.
The real advantage
To some extent, we are doing all this with current traditional technologies as well. More the data, Merrier it is, so big data technologies will definitely help, but is that all?
In my opinion, the real advantage of Big Data is to find the problem (or opportunity) that you do not even know about. When a data scientist dives deep into data and finds patterns and co-relates, there will be an Eureka moment, that will provide you the real ‘intelligence’ hidden in this data. Indeed, Big Data is a solution in search of a problem!
Friday, October 28, 2011
An ounce of knowledge is worth a ton of data
As my colleagues return from Insurance CIO summit and other conferences, I am getting bombarded with questions and suggestions of leveraging the BIG social data.. There are many ideas floating - targeted marketing, risk evaluation etc..
Well, I adopted words of Dr. Fayyad (Yahoo’s ex chief data officer) to reply back - An ounce of knowledge is worth a ton of data!
I think, the insurers will gain more, if they focus on generating 'insights' from the already available data, before looking at acquiring more data. There is a plenty of structured and un-structured data available within the premises of the organization, across several touch points.
How much that is being utilized? Do we have models to analyze the data and generate insights and predictions? Is this intelligence already integrated with the business processes - from customer acquisition, to underwriting and claims processing?
I think, we need to WALK before we RUN.
Well, I adopted words of Dr. Fayyad (Yahoo’s ex chief data officer) to reply back - An ounce of knowledge is worth a ton of data!
Notwithstanding the legal and moral issues, the availability of this data does not mean 'availability of knowledge'.The models put on this data are sometimes completely inadequate to generate any useful insight for insurers. Even if there are any, it is hard to tie back these insights to specific customers or prospects, thereby, making those completely non-actionable.
I think, the insurers will gain more, if they focus on generating 'insights' from the already available data, before looking at acquiring more data. There is a plenty of structured and un-structured data available within the premises of the organization, across several touch points.
How much that is being utilized? Do we have models to analyze the data and generate insights and predictions? Is this intelligence already integrated with the business processes - from customer acquisition, to underwriting and claims processing?
I think, we need to WALK before we RUN.
Sunday, June 19, 2011
Translating Business Strategy to Enterprise Architecture
I recently concluded a consulting assignment to define Future (after M&A) enterprise Architecture for a health insurance company, who acquired another company with considerable overlap in business.
While it was ‘relatively’ easier to come up future IT architecture by analyzing future needs and system overlap, it was quite challenging to present to executive board (completely non-technical with attention span of max 5 mins) and explain how exactly it maps to their business strategy. We had generated loads of detailed EA artifacts, however, challenge was to put all this together in just couple of slides and create a strong business case to move forward.
I found TOGAF’s Content Metamodel very useful in creating this linkage. I identified strategy business themes and for each business theme, and developed a view similar to content metamodel to link business strategy to required business services first, and then further to changes required in business process & IT systems.
A quick glance at artifacts is as shown -
While it was ‘relatively’ easier to come up future IT architecture by analyzing future needs and system overlap, it was quite challenging to present to executive board (completely non-technical with attention span of max 5 mins) and explain how exactly it maps to their business strategy. We had generated loads of detailed EA artifacts, however, challenge was to put all this together in just couple of slides and create a strong business case to move forward.
I found TOGAF’s Content Metamodel very useful in creating this linkage. I identified strategy business themes and for each business theme, and developed a view similar to content metamodel to link business strategy to required business services first, and then further to changes required in business process & IT systems.
A quick glance at artifacts is as shown -
Saturday, April 30, 2011
Keys for success in using a Global Delivery Model
My presentation at South New England PMI conference.
I shared thoughts and my experience of creating customized process framework by adopting best practices from traditional as well as agile methodologies that are appropriate for your project and your organization.
I shared thoughts and my experience of creating customized process framework by adopting best practices from traditional as well as agile methodologies that are appropriate for your project and your organization.
Monday, February 14, 2011
What if two Turkeys make an eagle?
If you follow ‘Mobile world’ news like I do, you might have already heard about partnership between Nokia and Microsoft, and Google’s trash-talk in response – “Two Turkeys do not make an Eagle”.
Cut to one year back – when android itself was a Turkey, however, they pretty much turned themselves into an Eagle. If you trust in Gartner’s figures, Android market share has risen from 3.5% in 2009 to 17% in 2010 and it is on its way to 22% this year. See - http://www.gartner.com/it/page.jsp?id=1434613 .
Nokia’s Symbian has highest share so far and having seen a world (e.g. India) completely dominated by Nokia phones, I believe they will put up a pretty serious fight for Apple and Google.
What does this mean to us in Insurance industry, who are developing mobile business apps? I think, these developments make a clear case for ‘cross-platform’ development. Rather than, making an application specifically for iPhone or Android, it’s time to seriously consider cross platform development platforms. A hybrid approach with combination of common ‘Portable’ code and some sexier ‘Native’ features appears as a good balance that provides functionality with oomph.
Cut to one year back – when android itself was a Turkey, however, they pretty much turned themselves into an Eagle. If you trust in Gartner’s figures, Android market share has risen from 3.5% in 2009 to 17% in 2010 and it is on its way to 22% this year. See - http://www.gartner.com/it/page.jsp?id=1434613 .
Nokia’s Symbian has highest share so far and having seen a world (e.g. India) completely dominated by Nokia phones, I believe they will put up a pretty serious fight for Apple and Google.
What does this mean to us in Insurance industry, who are developing mobile business apps? I think, these developments make a clear case for ‘cross-platform’ development. Rather than, making an application specifically for iPhone or Android, it’s time to seriously consider cross platform development platforms. A hybrid approach with combination of common ‘Portable’ code and some sexier ‘Native’ features appears as a good balance that provides functionality with oomph.
Subscribe to:
Posts (Atom)