The Data Quality Chronicle

An event based log about a service offering

Informatica v9 shifts and how to manage them: Source to Target


Introduction

I’ve recently made the shift from Informatica Data Quality / Data Explorer version 8.x to Informatica v9.  In the process I have discovered quite a few shifts in how certain tasks are performed.  Notice I called them shifts.  I did so because after learning them, I didn’t feel as though there were fundamental changes.

[tweetmeme source=”dqchronicle” only_single=false http:// http://bit.ly/9THbZJ]

A prime example and the focus of this post is the source-to-target process.  This process is so fundamental to ETL and data quality that it is often the starting point for all development.  As such I have chosen it as my starting point in mapping out the shifts in v9.

Shifting from the start …

In IDQ v8.x there was a source component that you added to the plan and defined either with custom SQL or by selecting the required data attributes.  In v9 you’ll have a hard time finding that component.  I know I did.  That’s because a shift has been made by Informatica away from source/target components and toward physical / logical data objects.  For simplicity purposes we’ll use physical data objects in our example.  However a logical data object is a construct that merits mention here and I’ll follow up with in a later posting.

Logically speaking

Briefly, a logical data object is a generic data object that is not customized to one particular data environment but rather is constructed to represent a typical instance of metadata.  An example is a logical data object that models addresses.  This data is fairly consistent in most environments, so it is advantageous to an ETL/DQ developer to build a generic object that represents the typical metadata associated with an address.  Once this object is built it can be defined as reusable and dropped into any mapping that requires address data.

In the physical world

Physical data objects are different than logical data objects in that they more directly represent the data environment as it exists in reality.  Physical data objects can be defined from an existing data source, such as a flat file or data store.  Once the physical data object is created it can be used as a read or write object.  Defining a physical data object as a read object is analogous to using the source component.

To create a read physical data object follow the steps defined below:

  1. Right-click on the Physical Data Object entry in the Data Explorer pane on the left-hand side of the Informatica Developer utility
  2. Select Create Physical Data Object from menu options provided
  3. Select either the flat file or data store source of the source data
  4. Be sure to double check the data types and precision of the attributes created from the source
  5. In the properties section of the data object specify run time properties like out file name and directory

Once the physical data object is created, drag-and-drop the object into a mapping and select the Read option from the dialog that appears.  Once this step is complete, you’ve made the shift to using data object instead of components to define sources in Informatica Data Quality v9!

I’ll cover write objects next month and move on to logical data objects shortly after, so stay tuned!

Please feel free to submit comments and questions and I will respond as quickly as possible …

Advertisements

9 responses to “Informatica v9 shifts and how to manage them: Source to Target

  1. John Muntges September 12, 2010 at 7:48 pm

    Very helpful, as I am just starting to work with version 9, and am starting to test our version 8 address validation routines.

    • William Sharp September 13, 2010 at 3:09 am

      Thanks for reading and commenting on the blog post, John. I have some content on here on v9 changes and plan to add more. V9 should provide a conversion utility to convert your v8 code to the v9 platform, if I am not mistaken. I’ll double check on that and get back to you.
      If you’d like to see a v9 topic covered, feel free to shoot me a note and I can do my best to cover it!

  2. Ranjeeta March 18, 2011 at 6:31 am

    Very Helpful William! Thanks for sharing u’r experience.

    • William Sharp March 18, 2011 at 12:42 pm

      Thanks for stopping by and commenting Ranjeeta! Glad I could shed light on the changes in v9 for you. Please come back and comment again soon.

  3. Ranjeeta March 18, 2011 at 6:33 am

    Pl. keep posting u’r experience with IDQ V-9

    • William Sharp March 18, 2011 at 12:42 pm

      Of course, Ranjeeta. Glad it is helpful!

  4. suma March 24, 2011 at 7:03 pm

    very informative..Please share some more exp if you have in Informatica like wt things should be taken care when shifting from V8 to V9 and at the same time server is changing to different server..

    Thanks!!

  5. Andy March 29, 2011 at 8:09 pm

    Did you ever do a blog post on using a physical database table as a write component? I am working with v9 and struggling to successfully write parsed data to my target table. I can successfully run a profile on the parsed component but after executing the mapping and attempting to view the data in the target table I get an error. The error is focused on the new target columns that I created.

    • William Sharp March 30, 2011 at 2:51 am

      Andy,
      I have not published a post but I am working right now on a project with the same requirements. I have a few questions for you so I can help:
      Did you review the log file of the mapping that is using the table write object?
      Are you loading the primary key of the table?
      Are you sure the data types of the mappings and the data types in the target table match for each attribute?
      Are you sure there is enough disk space to store the results in the table?
      Are you sure there is enough table space in the database to store the results in the table?
      I’d like to help you through this, respond and we’ll work through it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: