Supporting Your Data Management Strategy with a Phased Approach to Master Data Management 1

overview

For practical purposes, master data management is a set of disciplines and tools enabling business data consumers to access a unified view of shared data about one or more specific master data domains such as customer or product. Yet, while the technical processes for extraction and consolidation drive the activity in most MDM programs, the actual intent of MDM is to satisfy business consumer needs for access and delivery of consistent, shared data.

Satisfying user needs means “working backward” by understanding how master data is employed within critical business functions and processes – and how that data is to be consumed by the users. This means identifying the key data consumers in the organization and soliciting their needs and expectations (both now and in the future). Information architects must work with the business teams to understand how the organizational mission, policies and strategic performance objectives are related to the use of master data. Finally, it’s necessary to understand how improvements in information sharing will maximize corporate value.

A gap analysis is performed to determine what must be done to the current data environment in order to satisfy future data consumption objectives. Performing this gap analysis helps in two ways. First, it isolates key strategic goals for data sharing that must be put into place before any technical MDM approach can add value. Second, and more importantly, it establishes the value of fundamental data management best practices that benefit the organization beyond the needs of a specific master data project. Effective data management practices penetrate all aspects of information use across the enterprise, such as:

Data governance, which formulates the policies, processes and procedures to ensure that data use complies with explicit business policies; engages business data owners; identifies key data assets to be managed and shared; and delineates specific data requirements and quantifiable measures.

Metadata collaboration, which defines standards for business glossary terms and definitions, representations for conceptual data elements, and alignment of models in ways that will reduce conflicts when data sets are merged into a unified master view.

Data quality management, especially when deploying inspection and monitoring compliance with defined data standards and rules; and integrating services implementing data controls directly into the application infrastructure.

Integration of identity resolution and management within business process model, which best satisfies the ongoing need for maintaining unified views for specific entities such as customer or product.

Use of these best practices does more than lay the foundation for improved information. These practices highlight the relationship among business processes, information and use. They also emphasize the need to adjust staff member behaviors as access to master data provides greater customer and product data visibility. And while the success of your strategic management plan for MDM must have milestones and deliverables aligned with these disciplines, the organization will directly benefit from each practice area independently.

 

How Data Governance Supports the Data Strategy

Because enabling comprehensive visibility into a composed view of uniquely identifiable entities will continue to be part of the information strategy, there must be a reliable approach for:

  • Ensuring that proper validation and verification is performed and approved as new data is created or acquired.
  • Confirming that enterprise requirements for the quality of shared information are satisfied.
  • Accessing and managing the composed view of shared information within defined security controls.
  • Guaranteeing the consistency, coherence and synchrony of data views composed from a variety of sources.

Data governance provides the foundation for mapping operational needs to the framework of a sound data strategy designed around unified master data domains. There are numerous aspects of instituting a data governance program, including these key practices:

  • Data governance program management – Developing an ongoing program management plan that identifies roles, defines responsibilities, provides templates for key artifacts (such as data policies, data quality requirements and policy deployment plans) and specifies the types of tools to support the data governance program.
  • Data governance operating model – Specifying the organizational structure for operationalizing data governance, the interaction and engagement model to create a data governance council, the development of ongoing agendas, meeting schedules, and the establishment of measures to ensure that progress continues.
  • Definition and deployment of data policies – Developing a framework for the process and workflows related to drafting, reviewing, approving and deploying data policies, as well as integrating business rule validation within the application infrastructure.
  • Data stewardship – Defining the operational procedures for data controls, inspection, monitoring and issue remediation related to data policy violation.
  • Collaborative agreements. Introducing data governance and data stewardship opens the door for agreeing to existing or newly defined data standards, business glossary terms and definitions, data element concepts, and corresponding data types and sizes. A governance program facilitates these agreements, and collaboration tools can supplement the tasks associated with the metadata management life cycle.
  • Data lineage. Data lineage needs to be mapped from creation (or acquisition points) to the various modification or usage touch points across the application landscape. An accurate lineage map helps in understanding the business application and process impacts of modifications to definitions or underlying data structures. It also enables the insertion of data validity controls for inspecting and monitoring data quality and usability

A reasonable investment in metadata management can add value to the organization by facilitating communication about shared concepts across business functions, while reducing variance and complexity. Metadata management also adds value by smoothing the differences between data sourced from functional silos – and paves the way for an effective MDM effort.

Advertisements

Change of the WAN Landscape

Cloud services are at the center of growth for many enterprises. Currently, about 80 percent of U.S. companies are considering public or private cloud according to IDC, and many are actively building out hybrid cloud strategies. This growth in cloud usage is having a transformational impact on IT resource plans. A recent IDC survey revealed a flip in how IT budgets would be allocated over time. At the time of the survey, respondents were spending 58 percent of their IT budget on non-cloud architecture and 42 percent on cloud; they anticipated that usage changing to 44 percent non-cloud and 56 percent cloud in the next 24 months.

The research company also found that enterprises already using the cloud expected to increase cloud spending by 34 percent in the subsequent 24 months. Even more importantly, cloud is driving business growth. IDC forecasts that businesses will spend $122 billion on public cloud IT services in 2018.

This investment in cloud-based applications is spawning a new set of networking needs. Traditional WAN architectures were designed for efficiency and performance when applications primarily resided in the data center. With the proliferation of cloud and software-as–a-service (SaaS) applications, traditional means of connecting branches and users to applications are in need of a change. In short, businesses must look beyond the WAN-connectivity technologies currently in place, particularly Multiprotocol Label Switching (MPLS), to address their needs.

In addition to the need for new ways to connect users to SaaS and cloud-based applications, enterprises must also ensure the WAN delivers consistent performance across all sources of connectivity (e.g., DSL, Cable, LTE and MPLS), visibility and control for legacy and cloud-based applications, and faster service provisioning.

 

Times Are Changing

It’s been well over a decade since MPLS rose to prominence, replacing frame relay as the preferred WAN solution. MPLS’ reliability combined with its ability to deliver on SLAs helped drive its ascent. MPLS offered reliable access to data-center-based applications — which were predominant, provided support for branch-to-branch communication for voice and video, and could easily handle the small amount of non-critical Internet traffic that passed through the network.

However, in the past five years things have dramatically changed and continue to do so. Applications are moving to the cloud – the architecture, agility and flexibility that accompany such a transition do not favor MPLS, which is more rigid, expensive and not optimized for cloud application environments. For example, with MPLS, accessing a cloud-based application follows a very different path from accessing a data-

For example, with MPLS, accessing a cloud-based application follows a very different path from accessing a data-centerbased application. While MPLS provides branch users with direct access to an application housed in the data center, it can create a circuitous and more expensive path for branch users accessing cloud-based applications. In a cloud architecture with an MPLS-based WAN, the traffic must first travel over the MPLS network from the branch office to the data center before finally going out to the Internet, and then back the same route. This can negatively impact performance and cause costs to rise. According to IDC, 90 percent of new applications are being developed specifically for the cloud, and this gap will only continue to grow and render MPLS less effective. But

But cost isn’t the only issue. User experience has also become problematic. The challenges of using a traditional MPLS network to connect to cloud-based applications is often recognized when employees are frustrated by application performance at the office and find that accessing the same cloud-based applications from their home-based Internet connection is faster than at the office.

 

Looking Beyond MPLS

The questions enterprise IT consequently are asking are: Is there a way to leverage broadband for their enterprise WAN to make accessing cloud-based applications more efficient and less expensive? Can they introduce multiple sources of connectivity – MPLS, broadband, LTE and so on — without compromising the high level of reliability, security and performance they expect with their traditional WAN architecture? Finding a solution that combines the flexibility, scalability and cost of broadband with the control and reliability of MPLS seemed an impossible feat. Until now. Enterprises now have a solution called the software-defined WAN (SD-WAN). An SD-WAN resolves many of the shortcomings found in traditional WAN architectures by putting a secure, virtualized overlay on top of the WAN to enable simple, centralized provisioning, application and user visibility, and the use of multiple sources of connectivity simultaneously through dynamic multi-path control. More advanced SD-WAN solutions also deliver consistent performance regardless of the type of connectivity – all while driving down costs significantly. Essentially, an SD-WAN turns the WAN into a geographically distributed LAN, providing the enterprise with a dynamic solution that leverages multiple sources of connectivity, is faster to deploy and can be centrally managed. Gartner cites four key components of an SD-WAN solution. SDWANs:

1. Provide a lightweight replacement for traditional WAN routers and are agnostic to WAN transport (e.g., support MPLS, Internet and LTE).

2. Allow for load sharing of traffic across multiple WAN connections in an efficient and dynamic fashion that can be based on business or application policies.

3. Simplify the complexity associated with management, configuration and orchestration of WANs.

4. Must provide secure VPNs and have the ability to integrate additional network services

 

I.T. Performance and Business Repercussion

Businesses, both online and brick-and-mortar, are constantly updating their technologies to attract more customers and keep up with competition. While this approach can certainly be beneficial in customer acquisition, the process creates complicated issues for the IT department, which is already struggling against budget cuts and departmental downsizing. IT is a cost center where it’s expected to do more using less money. It is bogged-down by requests and demands for new features, a faster website, to scale infrastructure, and create an overall better product or service; meanwhile, the IT budget is among the first to be cut within a company and the average team size is dwindling. Considering the fact that there are less people and more work in IT Ops, time is a rare and valuable commodity; for this reason, digital performance analytics has evolved from a mere “insurance policy” for good measure to an absolute necessity for businesses.

There are two sectors of the costs associated with IT Ops: the cost of running the department eciently, and what your business stands to lose when performance failures occur. There’s a direct link between the performance of a website and its revenue. The cost of poor performance spans several dierent aspects of the business, including loss of transactions, loss of customer trust, SLA penalties, and extra resources required to resolve the issue.

Stats:

  • Walmart grew incremental revenue by up to 1% for every 100ms of improvement
  • Shopzilla decreased average page load time from 6 seconds to 1.2 seconds, which increased revenue by 12%, and page views by 25%
  • Amazon increased revenue by 1% for every 100ms of performance improvement
  • Yahoo increased trac by 9% for every 400ms of improvement
  • Mozilla got 60 million more Firefox downloads per year by making their pages 2.2 seconds faster

The average yearly amount lost by a company because of performance issues across North America and Great Britain was $140,003

USER EXPECTATIONS

User expectations are incredibly high and have been rapidly increasing as technology advances. Ensuring that your site is able to, at least, meet your users’ expectations is imperative to maximizing your bottom line.The indirect costs of poor performance can be detrimental to your business, and it only takes one bad user experience for your business to suer the consequences.

Stats :

  • 47% of users expect a webpage to load in 2 seconds or less
  • 64% of shoppers who are dissatisfied with their site visit will go to a competitor’s site for their next shopping experience
  • 88% of online consumers are less likely to return to a site after a bad experience
  • Almost 50% of users expressed a less positive perception of the company overall after a single bad experience
  • More than one-third of users would tell others about their disappointing experience

 

Converting json String to a Map, without Dependencies

Starting with JDK 8u60+ the built-in Nashorn engine is capable of converting JSON content into java.util.Map.

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Map;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;

public class JSONParsingTest {

    private ScriptEngine engine;

    @Before
    public void initEngine() {
        ScriptEngineManager sem = new ScriptEngineManager();
        this.engine = sem.getEngineByName("javascript");
    }

    @Test
    public void parseJson() throws IOException, ScriptException {
        String json = new String(Files.readAllBytes(/*path*/);
        String script = "Java.asJSONCompatible(" + json + ")";
        Object result = this.engine.eval(script);
        assertThat(result, instanceOf(Map.class));
        Map contents = (Map) result;
        contents.forEach((t, u) -> {
        //key-value pairs operations
        });
    }
}


Reffer Nashorn extensions documentation for more use cases.

Technology Stack at The Enterprise Level

The choice of technology stack at the enterprise level, especially when extending or expanding the offerings has lots of considerations:
* What does my tech ops know how to install, deploy, support? Introducing new technology is expensive.

* What does my existing staff know? Introducing new technology is expensive.

* What technology stack can I get indemnified libraries for? Introducing new technology is risky?

* What is the existing technology stack I need to interface with? Cross architecture connectivity can be expensive and suffer performance issues.

* What technology stack is used for the supported open-source solutions I might be interested in deploying? Bleeding edge can be expensive.

* What technology stack is used for the proprietary solutions I might be interested in deploying? Asking/paying for porting to my preferred stack is expensive.

* What does everyone else use? It’s politically safer to be a technology follower, especially with the high rate of software project failure.

* What makes programmers most efficient and effective? They are a major cost factor.

There are other considerations
* Technical fit – not likely to do browser plug-ins in COBOL.

* Maturity of the stack – what’s proven, secure, performing,

* The “–abilities”, support, scale, secure, maintain, development, port, upgrade, use, interopera, etc.?

* What does the CTO believe?

* What does the “though leaders” in the development organization believe?

* What does the company you are about to acquire use?

* What does the company about to acquire you use?

* What does your parent (or child) company use?

I’m sure I’m forgetting some, and clearly these all don’t carry the same weight. And, depending on the scope of the project, there’s different weights as well.

You get the picture. The choice is seldom based simply on the technical purity of the programming language. While that’s fun to debate, from a purely academic point of view, the real world is much messier.

Why Enterprises uses java? because enterprise apps are not about UI widgets only. Enterprise apps are running on multiple devices, supporting millions of concurrent users, with 99.99% of uptime.

So to achieve this its not about choosing the language only but the whole solution. Which includes Security, Persistence api, Transaction management, Clustering, Messaging, Webservices, ESB, Connectors for heterogeneous applications, Web support etc. You can see the reason why enterprise uses Java as its major programing language as JEE answers for most of these issues. Other than that there are many opensource solutions available and there is a big community behind it.

Enterprises are also using Javascript as a key component for their front end development regardless of what backend used. Specially for AJAX and SPA.

And if by Javascript you meant node.js, than we probably talking about the microservices and may be MEAN stack. which is a different application architecture. In that case you have opportunity to write Javascript both on server and client side, but even in that case there are still some Java components being used for integration.

Exposing REST Services Using Swagger

Swagger is a simple yet powerful representation of your RESTful API. With the largest ecosystem of API tooling on the planet, thousands of developers are supporting Swagger in almost every modern programming language and deployment environment. With a Swagger-enabled API, you get interactive documentation, client SDK generation and discoverability.

First thing you need to do when you use Swagger is use @Api annotation for describe endpoint.

Selection_061

Next thing need to add is @ApiOperation annotation. This will create a UI for the API.

Selection_062

Above annotation will create a following UI for you, You are building the documentation UI while you are creating the API. You don’t have to create a separate documentation.

Selection_063

Then you can add errors for rest api usage.

Selection_064

If some one wants to add specific value and test the api, you can implement it in documentation by @ApiParam annotation. All the details of the parameter can be describe in this doc too.

Selection_065

This is the full api code wrap with Swagger annotation.

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

import com.example.model.Account;
import com.example.service.AccountService;
import com.force.api.ApiException;
import com.wordnik.swagger.annotations.Api;
import com.wordnik.swagger.annotations.ApiError;
import com.wordnik.swagger.annotations.ApiErrors;
import com.wordnik.swagger.annotations.ApiOperation;
import com.wordnik.swagger.annotations.ApiParam;

@Controller
@RequestMapping(value = "/api/v1/account")
@Api(value = "Account operations", listingClass = "AccountController", basePath = "/api/v1/account", description = "All operations for accounts")
public class AccountController {
	
	@Autowired
	AccountService accountService;
	
	@ApiOperation(value = "Get all account (max:200)", notes = "Get all existing account names (max:200)", httpMethod = "GET", responseClass = "Account", multiValueResponse = true)
	@ApiError(code = 500, reason = "Process error")
	@RequestMapping(value = "/", method = RequestMethod.GET, produces = "application/json")
	public @ResponseBody List<Account> showAllAccounts() throws ApiException  {
	    return accountService.listAccounts();
	}
	
	@ApiOperation(value = "Get a existing account by Id", notes = "Get account name by specifying a Salesforce account id", httpMethod = "GET", responseClass = "Account", multiValueResponse = true)
	@ApiErrors(value = { @ApiError(code = 400, reason = "Invalid Id supplied"), @ApiError(code = 404, reason = "Account not found") })
	@RequestMapping(value = "/find/{accountId}", method = RequestMethod.GET, produces = "application/json")
	public @ResponseBody Account findAccountById(@ApiParam(internalDescription = "java.lang.string", name = "accountId", required = true, value = "string") @PathVariable String accountId) throws ApiException {
		return accountService.findAccountById(accountId);
    }
	
	@ApiOperation(value = "Delete a account giving an Id", notes = "Delete a account by specifying a Salesforce account id", httpMethod = "DELETE")
    @ApiError(code = 500, reason = "Process error")
    @RequestMapping(value = "/delete/{accountId}", method = RequestMethod.DELETE, produces = "application/json")
	public @ResponseBody String deleteAccount(@ApiParam(internalDescription = "java.lang.string", name = "accountId", required = true, value = "string") @PathVariable String accountId) throws ApiException {
		return accountService.deleteAccount(accountId);
    }
	
	@ApiOperation(value = "Create a new account", notes = "Creates a new Salesforce account by specifying a account name", httpMethod = "POST")
    @ApiError(code = 500, reason = "Process error")
    @RequestMapping(value = "/new", method = RequestMethod.POST, produces = "application/json")
	public @ResponseBody String createAccountFromParamName(@ApiParam(internalDescription = "java.lang.string", name = "name", required = true, value = "string") @RequestParam(required = true) String name) throws ApiException{	
		return accountService.createAccount(name);
    }
	
	@ApiOperation(value = "Update an exsiting account by Id", notes = "Update a existing account by specifying a Salesforce id", httpMethod = "POST")
	@RequestMapping(value = "/update/{accountId}", method = RequestMethod.POST, produces = "application/json")
	public @ResponseBody String updateAccount(@PathVariable(value="accountId") String accountId, @ApiParam(internalDescription = "java.lang.string", name = "name", required = true, value = "string") @RequestParam(required = true) String name) throws ApiException {
		Account updateAccount = new Account();
		updateAccount.setName(name);
		return accountService.updateAccount(accountId, updateAccount);
	}
}

This will create the following documentation UI.
Selection_066Lets try to create a new account and test the API. Click ‘try it out’ after providing the name parameter. Selection_067

It will shows the response body,code and any headers for the invoking above rest operation.
Selection_069I guess this would help you to get rough idea about Swager tool for API documentation. You can find complete project here

Obscure trends in open source

Annual Best of Open Source Software Awards, affectionately known as the Bossies , was held few days back. As you browse through the Bossies, one thing that will strike you is how essential so much of open source software has become to modern computing, particularly in application development and IT infrastructure. According to Bossies awards these are the trends  relatively obscure to the world at large.

1. Open source is ground zero for technology development. Once software vendors would open source software that, to put it kindly, wasn’t worth monetizing anymore. Now open source has become the preferred way of germinating hot new technology, particularly for startups. Docker and Hadoop — and in particular their exploding ecosystems — are the most obvious examples of this, not to mention the parade of NoSQL and NewSQL databases.

2. The cloud is eating open source applications. Browse through the “best open source applications” section of the Bossies, and you’ll see that many have a SaaS or hosted option. Makes sense — even Microsoft Office 365 is a semi-cloud offering and its chief rival is clearly Google Apps. These days, many IT departments would like to avoid installing and maintaining applications locally when possible.

3. Big Internet companies are huge open source contributors. You probably know that Google came up with MapReduce and Yahoo cooked up Hadoop. Maybe you also know that Google developed the most popular JavaScript framework, AngularJS, and contributed cgroups to the Linux kernel, which eventually became Docker — today’s open source darling. But did you know that eBay, Facebook, LinkedIn, Netflix, and Twitter have each originated dozens of open source projects? Facebook, for example, is responsible for both Cassandra, one of the leading NoSQL databases, and React, a wildly popular JavaScript library.

4. GitHub is the center of the universe. The world’s leading code repository and versioning system, offered as a cloud service, now hosts 27 million projects. Sure, you’ll find most of the high-profile open source projects here — and/or at Apache. But the point is that GitHub has changed software development forever by answering a simple question: Why code it yourself if someone else has already done it and is willing to share under a liberal open source license? Odds are, you can find something close to what you need on GitHub.

5. Security has become a major sore point. Last year was a tough one for open source security. We all know the Heartbleed saga: A flaw in OpenSSL stood unaddressed for two years, and once discovered threw admins everywhere into a panic. Six months later, we were confronted with the nasty Shellshock bug, which had lurked in the open source Bash project since 1989. The rising importance and ubiquity of open source solutions make them big fat targets, so the industry has been forced to collaboratively fund projects like OpenSSL that were woefully underresourced.