One of the neat things about Informatica is its ability to re-use code. You can take advantage of this by writing generic mappings that you intend to run in different ways, perhaps with a slight variation on the SQL Override, for example.
We have a couple of mappings which we are doing this with. Our customer match/consolidation sessions are replicated into 8 pieces, each doing a subset of the alphabet. We use a parameter file in the workflow to tell each session which letter to process. This way, we only have to maintain one version of the mapping, and it gets called 8 different ways.
The parameter file we use has entries for each session which look something like this:
[s_work_CustMasterMatched_AtoB]
$$STREAM=AtoB
$$STREAM_BEGIN=A
$$STREAM_END=Bzzzzz
$PMSessionLogFile=s_work_CustMasterMatched_AtoB.log
This is handy, because if the volume of data is too large to process (and we run out of memory trying to process it), I can cut it in half by processing only A to Azzzzz. And I don't have to change the code.
In your session properties, be sure to set your log file name to $PMSessionLogFile - anything other than this, and you're not going to get the name of your logfile right. Informatica looks for this variable name specifically. You can't even be clever and use something like $logfile here. It just won't work - you'll get a log named $logfile.bin - and all sessions sharing that name will write to the same log. No good.
Also remember that if you modify the SQL override at the workflow level, and changes you make to the mapping won't be reflected in the workflow (since workflow changes override session changes, which override mapping changes). So, be diligent in how you make updates to these kinds of jobs.
Monday, May 18, 2009
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment