OBIEE 11g Up-Gradation – RPD Gotchas and Glitches


Objective of this thread is to walk through across several issues found in RPD post Up-gradation from 10g to 11g. Let’s first start with OBIEE 11g Warnings / Errors and then I quickly moved to other areas to explore other glitches …. Idea is to bring a valuable thread for target audience to include important topics rather the conventional one found over several other fellow Bloggers post !

Nice Oracle documentation is there to detail out the step by step up-gradation process…

http://www.oracle.com/webfolder/technetwork/tutorials/obe/fmw/bi/bi11115/upgrade/upgrade_to_11g.htm

Pretty simple huh?? Yes …. Until you know the unknown and erratic issues on your way soon … So hold your seat belt tight to fly across the unknown territory with several assumptions/concerns/endless issues/unpredictable behaviors and finally achieving the goal …successful Up-gradation 🙂

Repository Up-Gradation Issues

1)      There are lots of unusual warnings have been observed after up-grading the RPD to 11g version. Some of them are really nonsense. However as Oracle said that strong consistency checker is involved in the whole process to mitigate the future risk of enhancement resulting these number of warnings. Details of the warnings could be found at 11g Up-grade planning ref doc:

Refer to section: 1.2.1.1 Enhanced Repository Consistency Checking

http://docs.oracle.com/cd/E21764_01/bi.1111/e16452/bi_plan.htm

Each warning has some specific number by which we can call them. Let’s see each of them one by one:

A)     Warnings [39028] 

Resolution: Call interface checked by consistency checker to see how compatible it is with the underlying database specification and how much it is aligned with database features defined in repository. Typically we have seen after up-gradation the features retains what is in 10g. In that case, solution is to make it Default database feature against each Physical layer database. That will remove the warnings however make sure that none of your overriding database features will cause performance concern. Some of them in 10g which could be done to get performance benefit in terms of better utilization of joins. So be careful before making them default and if that is mandatory to gain the similar performance benefits then that needs to be retained even if that will throw warnings irrespective of Oracle’s recommendation.

B)      Warnings [39055]/[39054]

Resolution: Typically the join missing between the mentioned Logical Dimension and Facts. Adding the extra join will resolve your problem. Make sure it is necessary and could not cause impact in other areas or otherwise ignore if you are not going to extract the projects. It is not really mandatory to fix these warnings. We remove these warnings to get some lesser counts.

Out of this warning context, in terms of join conditions I have experienced issues with Physical layer join problems post up-gradation. Some of my join conditions were invalid and this has been observed particularly in Usage Tracking physical database. However the error it will throw is really not comprehensible and hence I removed the invalid physical joins and add them back and did similar for logical joins too. This might be one-off incident but that is true for me.

C)      Warnings [39057]

Resolution: The reason of this warning as per my analysis is , whenever you add inner joins/outer joins with logical tables in the single logical table source you definitely specify the join condition with attributes and it is not a mandate to map those logical columns with the column mapping. But this warning expects attributes participated in the join conditions should be part of the column mapping. Necessarily older designs either 7.8.5 or 10.1.3.4, it is not mandatory to be those columns in the column mapping. But here at-least the join condition column needs to be in the column mapping. I don’t know the rationale behind this extra check and I don’t know the other possible solutions to remove these warnings. But the above mentioned approach helpful for me and I am sure if you are unlucky enough but want to fix these warnings then you might have to do a lot of hard-work as one of our repository we fix around 550 of this kind of warnings … Really annoying …! Without much business or design benefit. Good luck 🙂

D)     Warnings [39059]

Resolution: Most unwanted and most confusing and problematic warnings. This warning emphasis some best practices to be adhered. Typically we have seen some snow-flaking across dimensions and leveling those dimensions in different level of hierarchy across logical table source resulting these problems. You seem to hit this problem due to incorrect and un-orthodox content aggregation leveling. However there might be several other angles and reasons too.As in the document it specified that  “a new consistency check warning (39059) highlights when a fact logical table source joins to a lower-level dimension logical table source, resulting in a potentially invalid join. If you get a 39059 warning after upgrade, verify that the join is as intended and does not result in incorrect double counting.”

Fixing those warnings is really a tedious jobs specially when my RPD already build in 7.8.5, migrated to 10g and then 11g as lot of developers mind-blowing design already enriched the RPD a lot 🙂 and while you don’t know the business reason there is no point of fixing those warnings as this caused the design changes in greater aspects and this is quite impossible if you are old enough with your RPD. Fixing this warning is good for health of the RPD for new development & indeed bad for developers but bit difficult and quite risky for older data models. For our case we left ~1500 warnings in our Repository as it is as we are not sure how much the data impact will be there in our ~650+ reports.As per Oracle ” If the join is as intended, then ignore the 39059 warning”

 Last but not the least

Apart from those warnings all errors should be fixed to have a Consistent repository and essentially you need to note that while there are 3 errors in repository your warnings count could be say 20 but after fixing those errors (which you must have to do) count can increase to 600 .Sound strange.. ! Really not , as when Consistency checker encounter error it skips showing the warnings and insist you to fix the errors before it proceed with further level of checks ! So best of luck. One more things don’t worry much about the warnings … even if there are your repository is consistent and none of the reports (apart from the complex one in catalog specially the combined requests) will show incorrect results. We have experienced this and you can easy to go ahead with warnings …. Let’s keep the life simple …. Now lets see what Oracle made for us to lead simple life …

Essentially Oracle has faced lot of questions with this mind-blowing checker (!) (yes I mean it) and they release a patch for OBIEE 11g (11.1.1.3) to hide those warnings(not errors) but not yet for 11.1.1.5. They are yet to fix this in future version 11.1.1.6, if any. See Oracle Doc ID 1365964.1 [How to Hide Selected Consistency Check Warnings in BIEE 11G]

Patch 11810367

As of when I am writing this thread for BIEE 11.1.1.5 Development teams are working on a Patch for BIEE 11.1.1.5. Please open an SR with BIEE Support to check if the Patch is available for your platform or not. This issue will be fixed in BIEE 11.1.1.6.

2)      During up-gradation the Upgrade Assistant migrate the Security groups / roles and users from Repository to Weblogic security policy store and hence it is the best practice to up-grade the repository at-least irrespective of whether you upgrade the catalog at all. But if you think of to up-grade the catalog first then you will definitely lose all the Catalog groups and permissions setup inside your Catalog objects even if you up-grade the RPD next time. So upgrade either RPD or RPD and Catalog both and never Catalog alone.  You will hit a lot of unforeseen behavior post up-gradation with catalog privileges as it has been observed that the permissions are not propagated properly in the online catalog.

3)      Job Manager “Administrator” Password is not upgraded with RPD upgrade .It still retains the password of source RPD and you need to change it manually.

4)      Usage Tracking (S_NQ_ACCT) and Scheduling objects (S_NQ_) moved from the target database area to RCU schema and the object definition itself changed like new column added and several column types changed. So ideally it is better to copy the 11g provided Usage tracking RPD layers and merge it with your upgraded RPD .However in that case all the Usage Tracking customization needs to be properly applied too.

5)      “Act As Proxy User” functionality no more going to work with Administrator while this user is part of Weblogic Default Authenticator scheme. Ideally this Administrator user-created by default in every Weblogic Default authentication scheme as it is automatically migrated from RPD to WLS security policy. Considering your RPD has authentication scheme set to LDAP in that scenario “ACT As” is no going to work with “Administrator” as dual switching of mixed Authentication scheme not possible (in this case one is WLS authentication and another is RPD LDAP authentication).So any of the LDAP user could Act as proxy users for other LDAP users and thus authentication scheme belongs to one area only. Otherwise you will see that Act As will not populated the list of proxy users and the list will be NULL or blank.

Another way will be to define the LDAP in Weblogic itself and maintain the Administrator user in the LDAP tree so that you can use Administrator to ACT AS proxy for other LDAP users.

So either all should be LDAP of Weblogic Default Authenticator or LDAP of RPD only at a time.

6)      Post upgrade Repository could not be opened online using the Administrator password .Resolution is perform the refresh GUID activities. You will get below error in log file:

[nQSError: 13041] The GUID of user ‘Administrator’ does not match user reference GUID at the repository. Please ask the administrator to delete the old user reference at the repository and login again. The process is explained under Oracle Doc ID 1265802.1. There might be several blog in the net which already listed down the steps.

7)      In our implementation we have seen that upgraded 11g repository crashes the environment while user try to log in .However this behavior not observed for all users. Digging into details our finding is that “Log Level” value populated from external table authorization should not be NULL otherwise 11g repository will crash your environment. So make sure the loglevel field must be updated at-least with some numeric value.

8)      While Saving Online RPD Changes in the Administrator Tool of OBIEE 11g Throws “[nQSError: 37005] Transaction update failed.” This is an Unpublished bug 9884975 – 11g admin tool receives nqserror: 37005 when saving online rpd changes

9)      During check-in and saving the repository we have encountered getting error : [46036] internal assertion: condition m_lock.isacquired() . For both 8) and 9) apply Patch 12909840

10)      Issue exist with Presentation column level security default access and it is not working properly with the PROJECT INACCESSIBLE COLUMN is NULL settings of NQSCONFIG.INI.Get the patch from Oracle to get rid of this issue.

11)      I have seen several connection related problem from RPD to database and most of the errors are Unable to connect the database. This happens due to Oracle Native OCI call and Middleware Native OCI conflict. As a best practice I usually follow below rule:

Copy the same tnsnames.ora across Oracle Home and Middleware home .Then create 3 variables in Windows : ORACLE_HOME , BI_ORACLE_HOME , ORACLE_BI_HOME as Middleware Home directory and then ORACLE_INSTANCE as Middleware instance directory . Bounce all services and you will be clean with any connection.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I hope above information is helpful for your 10g -) 11g Up-gradation process . Then don’t forget to rate this thread as above with Stars ! 🙂

Stay tuned for more update on Enviroment/Configuration Issues and BI Catalog Issues ….

Latest update for those available at: https://debaatobiee.wordpress.com/2012/05/14/obiee-11g-up-gradation-issueschallenges-11-1-1-5-and-6-too/

Cheers , DxP

OBIEE 11g Design Tips – The Best Practices


I had already composed the OBIEE design guidelines and best practices in my previous quite old post which mostly encircled the approach and best adopted strategies for previous release of OBIEE (aka Siebel Analytics 7.8.5) and OBIEE 10.1.3.4 version. Refer to my previous post :

https://debaatobiee.wordpress.com/category/obiee/best-practices/

Here are the design tips and guidelines to adhere as per Oracle recommended Metadata Repository Builders guide of OBIEE11g . The excerpt from it directly compiled like below:

Design Tips for the Physical Layer

  • It is recommended that you use table aliases frequently in the Physical layer to
    eliminate extraneous joins, including the following:
    – Eliminate all physical joins that cross dimensions (inter-dimensional circular
    joins) by using aliases.
    – Eliminate all circular joins (intra-dimensional circular joins) in a logical table
    source in the Physical Model by creating physical table aliases.
    For example, say you have a Customer table that can be used to look up
    ship-to addresses, and using a different join, to look up bill-to addresses.
    Avoid the circular joins by aliasing the table in the Physical layer so that there
    is one instance for each purpose, with separate joins.
  • You might import some tables into the Physical layer that you might not use right
    away, but that you do not want to delete. To identify tables that you do want to
    use right away in the Business Model and Mapping layer, you can assign aliases to
    physical tables before mapping them to the business model layer.
  • An opaque view (a Physical layer table that consists of a SELECT statement)
    should be used only if there is no other solution to your modeling problem.
    Ideally, a physical table should be created, or alternatively a materialized view.
    Opaque views prevent the Oracle BI Server from generating its own optimized
    SQL, because they contain fixed SQL statements that are sent to the underlying
    data source.

Design Tips for the Business Model and Mapping Layer

  • Create the business model with one-to-many logical joins between logical
    dimension tables and the fact tables wherever possible. The business model
    should ideally resemble a simple star schema in which each fact table is joined
    directly to its dimensions.
     Every logical fact table must join to at least one logical dimension table. Note that
    when the source is a fully denormalized table or flat file, you must map its
    physical fact columns to one or more logical fact tables, and its physical dimension
    columns to logical fact tables.
  • Every logical dimension table should have a dimensional hierarchy associated
    with it. This rule holds true even if the hierarchy has only one level, such as a
    scenario dimension {actual, forecast, plan}.
  • Make sure that all appropriate fact sources map to the proper level in the
    hierarchy using aggregation content. You set up aggregation content in the Levels
    tab of the Logical Column dialog for the measure. Note that this is different from
    the Content tab of the Logical Table Source dialog, which is used to specify the
    grain of the source tables to which it maps.
  • Typically, logical fact tables should not contain any keys. The only exception is
    when you need to send Logical SQL queries against the Oracle BI Server from a
    client that requires keys. In this case, you need to expose those keys in both the
    logical fact tables, and in the Presentation layer.
  • Normally, all columns in logical fact tables are aggregated measures, except for
    keys required by external clients, or dummy columns used as a divider. Other
    non-aggregated columns should instead exist in a logical dimension table.
  • In some situations, you might want to have multiple logical fact tables in a single
    business model. For Logical SQL queries, the multiple logical fact tables behave as
    if they are one table. 
  • You can define calculations in either of the following ways:
    – Before the aggregation, in the logical table source. For example:
    sum(col_A *( col_B))
    – After the aggregation, in a logical column derived from two other logical
    columns. For example:
    sum(col A) * sum(col B)
    You can also define post-aggregation calculations in Answers or in Logical SQL
    queries.
  • If you plan to use Oracle Scorecard and Strategy Management, it is a best practice
    to implement at least one time dimension in the Oracle BI repository you are using
    for your KPIs. This action is necessary because you use KPIs in scorecards to
    measure progress and performance over time. Note that an individual scorecard
    automatically picks up any dimension used by KPIs in that scorecard.
  • Aggregate sources should be created as separate logical table sources. For fact
    aggregates, use the Content tab of the Logical Table Source dialog to assign the
    correct logical level to each dimension.
  • Each dimension level in a hierarchy must have a unique level key. Also, each
    logical dimension table must have a unique primary key. Normally, this key is
    also used as the level key for the lowest hierarchy level.
  • Renaming columns in the Business Model and Mapping layer automatically
    creates aliases (synonyms) for Presentation layer columns that have the property
    Use Logical Column Name selected.
  • To prevent problems with aggregate navigation, ensure that each logical level of a
    dimension hierarchy contains the correct value in the field named Number of
    elements at this level. Fact sources are selected on a combination of the fields
    selected as well as the levels in the dimensions to which they map. By adjusting
    these values, you can alter the fact source selected by the Oracle BI Server. See
    “Creating Logical Levels in a Dimension” for more information about setting this
    value.

 Modeling Outer Joins
The following guidelines provide tips on how to model outer joins:

  • Due to the nature of outer joins, queries that use them are usually slower. Because
    of this, define outer joins only when necessary. Where possible, use ETL
    techniques to eliminate the need for outer joins in the reporting SQL.
  • Outer joins are always defined in the Business Model and Mapping layer. Physical
    layer joins do not specify inner or outer.
  • You can define outer joins by using logical table joins, or in logical table sources.
    Which type of outer join you use is determined by whether the physical join maps
    to a business model join, or to a logical table source join.
  • Be aware that outer joins in logical table sources are always included in a query,
    even if the none of the columns in one of the mapped physical tables are used.
    For example, assume that a logical table source is mapped to physical tables A and
    B. When no outer joins are defined, if physical table A is not required to satisfy a
    request, it is not included in the physical query. However, if the logical table
    source has an outer join defined to table A, table A is still included in the physical
    query, even if only physical table B is required to satisfy the request.
  • If you must define an outer join, try to create two separate dimensions, one that
    uses the outer join and one that does not. Make sure to name the dimension with
    the outer join in a way that clearly identifies it, so that client users can use it as
    little as possible.

 Design Tips for the Presentation Layer

  • You can show fewer columns than exist in the Business Model and Mapping layer.
    For example, you can exclude the key columns because they have no business
    meaning.
  • You can organize columns using a different structure from the table structure in
    the Business Model and Mapping layer.
  • You can display column names that are different from the column names in the
    Business Model and Mapping layer.
  • You can set permissions to grant or deny users access to individual subject areas,
    tables, and columns.
  • You can export logical keys to ODBC-based query and reporting tools.
  • You can create multiple subject areas for a single business model.
  • You can create a list of aliases (synonyms) for presentation objects that can be used
    in Logical SQL queries. This feature lets you change presentation column names
    without breaking existing reports.
    The following is a list of tips to use when designing the Presentation layer:
  • Because there is no automatic way to synchronize all changes between the
    Business Model and Mapping layer and the Presentation layer, it is best to wait
    until the Business Model and Mapping layer is relatively stable before adding
    customizations in the Presentation layer.
  • There are many ways to create subject areas, such as dragging and dropping the
    entire business model, dragging and dropping incremental pieces of the model, or
    automatically creating subject areas based on logical stars or snowflakes. See
    “Creating Subject Areas” for information about each of these methods. Dragging
    and dropping incrementally works well if certain parts of your business model are
    still changing.
  • It is a best practice to rename objects in the Business Model and Mapping layer
    rather than the Presentation layer, for better maintainability. Giving user-friendly
    names to logical objects rather than presentation objects ensures that the names
    can be reused in multiple subject areas. Also, it ensures that the names persist
    even when you need to delete and re-create subject areas to incorporate changes to
    your business model.
  • Be aware that members in a presentation hierarchy are not visible in the
    Presentation layer. Instead, you can see hierarchy members in Answers.
  • You can use the Administration Tool to update Presentation layer metadata to give
    the appearance of nested folders in Answers. See “Nesting Folders in Answers” for
    more information.
  • When setting up data access security for a large number of objects, consider
    setting object permissions by role rather than setting permissions for individual
    columns. See Chapter 13, “Applying Data Access Security to Repository Objects”
    for details.
  • When setting permissions on presentation objects, you can change the default
    permission by setting the DEFAULT_PRIVILEGES configuration setting in the
    NQSConfig.INI file.