java.net.SocketTimeoutException in OBIEE/Weblogic : Read timed out

I can continue write a long thread about this issue that we have faced in regard of this SocketTimeoutException. However this time I will keep in concise.
Here is the issue : two of the prompts in our Production environment suddenly broken which was working fine for past 3-5 months. In between we have made bundle patch upgrade on our 11.1.1.7 env , we did network/firewall level changes so we were not sure why those two prompts started behaving weirdly. Also we found we had similar kind of prompt exist elsewhere which was not facing the vulnerabilities.

By broken, I mean when user click on the prompt they are keep getting “Please wait” appearing in the drop-down menu and the drop-down value never displayed. And this only happens when you are accessing application through web / outside your corporate network and through external DNS. The prompt works fine in internal VPN network.
Since we have thousands of login happen each day, the Production OBIEE environment crashes after couple of days because of this issue. By crashes we mean OBIEE/EM?Console login hangs , you will not find any process went down in opmnctl status -l , all alive , EM / Console even doesn’t show anything red(if you able to login here) determining its a crash .

Pretty interesting ….. !!! Huh ?

When we inspect the console log of browser we have spotted after 2-3 minutes the browser throws below error :

“Internal Server error” Ref :
500 Internal Server Error
ERROR Codes for Reference #3.5e7f1cb8.1547495087.4c779890

The above was Akamai reference number. And for this issue there is nothing to do with Akamai edge or origin server and its cache issue. We have checked Akamai has no issue with cache content

As soon we receive below error the ADR incident log for obiee , bi_Serverx.log and biserver_diagnostic.log throws below :

This appears in Incident log (change the path to be exact for your environment)
+++++++++++++++++++++++++++++
/u00/app/Middleware/user_projects/domains/bifoundation_domain/servers/bi_server1/

adr/diag/ofm/bifoundation_domain/bi_server1/incident/incdir_539
Incident Id: 7997
Incident Source: SYSTEM
Create Time: Tue Jan 15 12:11:11 EST 2019
Problem Key: DFW-99998 [java.net.SocketTimeoutException][oracle.security.jps.ee.http.JpsAbsFilter$1.run][analytics]
ECID: e8398b29c4083075:10e4302f:1685274d016:-8000-0000000000008333
Application Name: analytics
User Name: <WLS Kernel>
Error Message Id: DFW-99998

Context Values
————–
DFW_SERVER_NAME : bi_server1
DFW_DOMAIN_NAME : bifoundation_domain
DFW_USER_NAME : <WLS Kernel>
DFW_APP_NAME : analytics
Description
———–
Incident detected using watch rule “UncheckedException”:
Watch time: Jan 15, 2019 12:11:11 PM EST
Watch ServerName: bi_server1
Watch RuleType: Log
Watch Rule: (SEVERITY = ‘Error’) AND ((MSGID = ‘WL-101020’) OR (MSGID = ‘WL-101017’) OR (MSGID = ‘WL-000802’) OR (MSGID = ‘BEA-101020’) OR (MSGID = ‘BEA-101017’) OR (MSGID = ‘BEA-000802′))
Watch DomainName: bifoundation_domain
Watch Data:
DATE : Jan 15, 2019 12:11:11 PM EST
SERVER : bi_server1
MESSAGE : [ServletContext@756918633[app:analytics module:analytics path:/analytics spec-version:2.5 version:11.1.1]] Root cause of ServletException.
java.net.SocketTimeoutException: read is alrady timed out
——————–

THIS APPEARS IN BI_SERVERx.log
/u00/app/Middleware/user_projects/domains/bifoundation_domain/servers/bi_server2/logs/bi_server2.log
++++++++++++++++++++++++++++++
[WARNING:7] [WL-320068] [Diagnostics] [host: <hostname>] [nwaddr: [10.30.xx.xxx] [tid: [ACTIVE].ExecuteThread: ’71’ for queue: ‘weblogic.kernel.Default (self-tuning)’] [userId: <WLS Kernel>] [LOG_FILE: /u00/app/Middleware/user_projects/domains/bifoundation_domain/servers/bi_server2/logs/bi_server2.log] Watch ‘UncheckedException’ with severity ‘Notice’ on server ‘bi_server2’ has triggered at Jan 23, 2019 9:36:00 AM EST. Notification details: [[
WatchRuleType: Log
WatchRule: (SEVERITY = ‘Error’) AND ((MSGID = ‘WL-101020’) OR (MSGID = ‘WL-101017’) OR (MSGID = ‘WL-000802’) OR (MSGID = ‘BEA-101020’) OR (MSGID = ‘BEA-101017’) OR (MSGID = ‘BEA-000802′))
WatchData: DATE = Jan 23, 2019 9:36:00 AM EST SERVER = bi_server2 MESSAGE = [ServletContext@1881523291[app:analytics module:analytics path:/analytics spec-version:2.5 version:11.1.1]] Root cause of ServletException.
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:128)
at weblogic.servlet.internal.PostInputStream.read(PostInputStream.java:196)
at java.io.InputStream.read(InputStream.java:82)
at weblogic.servlet.internal.ServletInputStreamImpl.read(ServletInputStreamImpl.java:222)
at com.siebel.analytics.utils.InputStreamWithLimit.read(InputStreamWithLimit.java:39)
at com.siebel.analytics.utils.IOUtils.copyStreams(IOUtils.java:66)
at com.siebel.analytics.web.j2eeutils.SAWHttpUtils.getRequestBytes(SAWHttpUtils.java:39)
at com.siebel.analytics.web.integration.LoadBalancerHTTPFilter.determineServerFromRequest(LoadBalancerHTTPFilter.java:293)
at com.siebel.analytics.web.integration.LoadBalancerHTTPFilter.determineServerFromRequest(LoadBalancerHTTPFilter.java:243)
at com.siebel.analytics.web.integration.LoadBalancerHTTPFilter.doFilter(LoadBalancerHTTPFilter.java:175)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
at oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:119)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:324)
at oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:460)
at oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:103)
at oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:171)
at oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:71)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
at oracle.dms.servlet.DMSServletFilter.doFilter(DMSServletFilter.java:163)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3748)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3714)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2283)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2182)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1499)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:263)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
SUBSYSTEM = HTTP USERID = <WLS Kernel> SEVERITY = Error THREAD = [ACTIVE] ExecuteThread: ’38’ for queue: ‘weblogic.kernel.Default (self-tuning)’ MSGID = WL-101017 MACHINE = <hostname> TXID = CONTEXTID = 0000MXv8etiFKAw_wDCCyW1SGacg003qNu TIMESTAMP = 1548254160834
WatchAlarmType: AutomaticReset
WatchAlarmResetPeriod: 30000
]]

++++++++++++++++++++++++++++

As you are seeing there are two variations of the log :

1) java.net.SocketTimeoutException: Read timed out
2) java.net.SocketTimeoutException: read is alrady timed out (note: the typo is not me and it is writing by the OBIEE product itself ! )

Essentially both are same and incident log says this is your problem key : “Problem Key: DFW-99998 [java.net.SocketTimeoutException][oracle.security.jps.ee.http.JpsAbsFilter$1.run][analytics]”

When we started investigating we have seen the prompt code is not special and we have used regular SQL results with SELECT statement and UNION statement.
Most weird part of if we use union all (as lowercase) it works and if we use FETCH ONLY 650001 ROWS clause at the end of the select statement of prompt logical query it works absolutely fine. Again these are the interesting facts which we can contradict later but these are our findings.

Here is the solution :

After going through several layers of network trace , Akamai , IDP Ping SSO , External / Internal DNS check, Checking the packets transfer across firewall , tracing the various logs we are seeing that when you click prompt the request headers even not passing through presentation server session log so its being stuck somewhere in the network and network is not able to process the request . Finally we came to know that we had security layer incorporated by Secureworks (that is a Intrusion Prevention System) which intercept the OBIEE browser POST request header and when it sees the SQL Injections operation (by SELECT or UNION ) via its Intrusion Prevention rule it blocks the inflow of traffic and that stops passing the request to other network layers and cause Weblogic server to go in unknown state. We had to allow those SQL rules across network as white listed traffic to stop this “SocketTimeoutException”.
As soon as we did this those 2 prompts started working fine but the “SocketTimeoutException” didn’t completely goes away . However the volume of this Exception reduced by 100x, 200x and we have seen no sign of crashing the OBIEE platform.

So in Summary :

Root Cause:

The Intrusion Prevention System misidentifying application traffic as malicious.

 Contributing factors:

• OBIEE uses raw SQL statements for normal application functionality causing the IPS to misidentify the traffic as malicious.
• OBIEE products do not utilize TLS.
• Specific filter in the OBIEE, using those filters generate the socket timeout exception which create cascading effect having the product to become unresponsive
• The server encountered an internal error or mis-configuration and was unable to complete the request.

Unfortunately, because of this Network issue, being in application team, we had to loads of hours to do root cause analysis before we figure this out. Facts of life for developers !!!

Authentication error while upgrading to JDK8 on 11g weblogic 10.3.6.0

Upgrading JDK for existing 11g OBIEE is very simple as it is defined in Oracle Documentation here:  https://docs.oracle.com/cd/E28280_01/bi.1111/e10541/patching.htm#BIESG9065

Generally if not said explicitly in Certification matrix it is assumed that any higher jdk6+ will be compatible with OBIEE 11g which is not in this case. That is weird !

But looks like its not that simple when you have followed the process and all services (Admin / Managed / Obis / Obips / opmn) up and running but heck no clue (not even from log) why the the user authentication fails.

The only clue can be given by weblogic Console when you launch it and go to Deployment components you will several weblogic components is in “Prepared” state rather “Active” . So this is your problem .

When you search for this error you would be able to know sun/io/CharacterEncoding is obsolete for JDK 8 and some of the dependent process will fail subsequently .

Now the only solution you would be having to resolve this problem to downgrade to JDK 7. Why I have faced this error is, I want to execute my bi-migration tool jar exported from JDK 8 (as in OBIEE 12c) and want to create a 11g export bundle for use in 12c as import archive. After making the JDK change I started all the services and want to see if my application working fine or not and found its broke.

 

OBIEE cool effect – Mouse hover on table and row highlight

My requirement was to add some cool effect on table row hover when you move your mouse across regular table or pivot table view in OBIEE

The advantage of this feature is, its cool and not distracting and it will help you to focus one row of important analysis which you can easily distinguish from the other rows.

Here is how it looks like once you hover . It changes the table row background color to light yellow.

Also here is the look and feel on alternate row colors(as Cyan) and hover on one particular row as Yellow.

Here is the simple CSS code you need to add in Narrative view of the report and in the compound layout where your table/pivot table view is placed. Below code works good for both Table and Pivot table view

&lt; style &gt;

.PTChildPivotTable tr:hover td {
background-color: #FFF1CD !important;
}

&lt; /style &gt;

Just replace &lt; with < and &gt; with > . So what it is doing is finding the Table class used in OBIEE and overriding the table row (tr) hover on table data (td) and changed the color.

Simple and sweet ! and isn’t it cool ? 😛

 

OBIEE Popup / Tooltip / Mouse Hover effect

There are some thread/blogs already talks about how to get the OBIEE popup/tooltip effect . As some of the threads are not easy to understand and for some blogs the popup/tooltip is not working for me I want to write my own.

As we know this feature is cool but unfortunately this feature is not available as out of box with OBIEE 11g or 12c.

This leverages a JS called Tipped JS and leveraging the JQuery and Javascript capabilities out there.

So this is the requirement :

We have an Event Main report which shows the aggregated metrics for the number of attendees in that event and we want to mouse hover on Event name and want to see the details as in popup/mouse hover/tooltip report exactly like below:

So I need to create 2 different reports. One is Main report and other is detail. Idea is every time you hover on Event Name in Main report it is going to run additional query to run Event detail report. So for OBIEE each hover is new query . This is very cool and very powerful to run the detail report on each mouse hover. And at the same time DB cost is associated for that. To get a really nice cool hover effect the DB needs to return the results very fast.

So you need custom JS to give that tooltip effect from here : http://www.tippedjs.com/ and will use a powerful OBIEE feature which is HTML expression wrapped around to insert JS , JQuery , CSS , HTML and OBIEE GO URL code  to format the look and feel.

Pushed that required JS library to OBIEE and restart analyticsRes module from console. So in my case I copied the files which packaged inside tipped JS here and update analyticsRes / Stop and start that deployment module :  [MiddleWare_Home]/instances/instance1/bifoundation/OracleBIPresentationServicesComponent/coreapplication_obips1/analyticsRes/tipped-3.2.0.1-demo

drwxrwxr-x 5 orabi orabi 4096 Jan 8 2014 js
drwxrwxr-x 3 orabi orabi 4096 Jan 8 2014 example
drwxrwxr-x 3 orabi orabi 4096 Jan 8 2014 css
-rw-r–r– 1 orabi orabi 93068 Aug 5 2016 jquery-1.9.0.min.js

Next, you need to refer those library as static text in your main report and add below:just replace &lt; with < and &gt; with >

&lt ; script type=”text/javascript” src=”http://code.jquery.com/jquery-1.10.2.min.js&#8221; &gt; &lt ; /script &gt;

&lt ;script type=”text/javascript” src=”http://obieenode:9704/analyticsRes/tipped-3.2.0.1-demo/js/tipped/tipped.js”&gt; &lt ; /script &gt;

&lt ;link rel=”stylesheet” type=”text/css” href=”http://obieenode:9704/analyticsRes/tipped-3.2.0.1-demo/css/tipped/tipped.css”/ &gt;

And add below as column expression in the column in your report : In my case I have added it under “Event Name” expression : (ignore the error after hitting ok) . Refer the screenshot below incase text is garbled with wordpress formatting.

‘&lt;div id=”showme’ || TRIM(CAST(“D_EVENTS_DXP”.”EVENT_ID” AS CHAR(10))) || ‘”&gt;’ || D_EVENTS_DXP.EVENT_NAME || ‘&lt;/div&gt;’ || ‘&lt;script type=”text/javascript”&gt; jQuery(document).ready(function($) {Tipped.create(“#showme’ || TRIM(CAST(“D_EVENTS_DXP”.”EVENT_ID” as CHAR(10))) ||'”, “saw.dll?Go&path=%2Fshared%2FPOC%2FEvent_Description&Action=extract&p0=1&p1=eq&p2=%22D_EVENTS_DXP%22.%22EVENT_ID%22&p3=’ || TRIM(CAST(“D_EVENTS_DXP”.”EVENT_ID” AS CHAR(10))) || ‘”, {ajax: true, skin:”blue”, hook: “topright”, border: { size:1}, afterUpdate: function() {} } ); }); &lt;/script&gt;’

And mark HTML in Data format:

So here is the explanation of doing this:

‘showme’ is just a div placeholder.

Event ID is the value that I want to pass to GO URL detail report which will show content on hover as popup/tooltip

Event Name is where I want my mouse hover to be placed

I call JQuery ready function to execute actions once the page is completely loaded

I call Event_Description detail report and user parameter called Extract to show the data immediately while passing EVENT_ID as another parameter. A very common mistake here is CASTING and TRIM . I found that your number(EVENT_ID value) doesn’t work well with character (EVENT_NAME value) so I had to user CAST and TRIM . If you don’t do that your popup will NOT work (at least it my case) . This is different than what is other blogger already wrote.

Next is AJAX call for changing the skin (the background color what you want). In my case its blue.

So that’s it .  Sweet and simple and easy . This solution should work in OBIEE 12c as well. Be very careful when you want to implement it as this can cause huge DB performance impact if you hover a lot across table grid.

Feel free to leave me message if I can help.

 

 

 

Change OBIEE prompt label font/color

In OBIEE there is no out of box option to change the Prompt label text color/font/type etc etc . However with this hack you can turned any prompt label in the style/type/family and color you want . This should be work fine with OBIEE 11g and 12c both version.

Assume there is a Prompt label called ‘Month’ and I want to change font color to Red . Just add below code as in “Text section” in dashboard layout and turn “HTML markup” on. Make sure the code is like in image with script tag open and closed.

————————————

https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js

$(document).ready(function(){
$(“span”).each(function(){
if($(this).text() == ‘Month’){
$(this).css(‘color’,’#ff0000′);
}
});
});

————————————

This is how you got the prompt color changed to red .

Hiding the OBIEE top banner on click button

Requirement is to hide OBIEE top banner because sometime top banner take extra or unnecessary vertical spaces which client may not like.

Here is what is present :

So clicking on button “Hide top Pane” should hide the OBIEE top banner .

Just add new text section in dashboard layout and mark as HTML markup. Add below JS code:

This will work like toggling through the button and once you click it will hide top pane and click again to show it again.

var isHidden = false;
function loadToggleAction() {
var sheet = document.createElement(‘style’)
if(!isHidden){
sheet.innerHTML = “.HeaderContainer {display:none;}”;
}else{
sheet.innerHTML = “.HeaderContainer {display:block;}”;
}
document.body.appendChild(sheet);
isHidden = !isHidden; //This will change the value to the opposite
}

<form>
<input type=”button” id=”dxp” class=”button” value=”Hide top Pane” onclick=’javascript: loadToggleAction();’ />
</form>

See the image how it should be : (add script tag and end script tag in proper place)

Customize OBIEE HeaderNavBar- Font/Color

As you know its very easy to customize the OBIEE Top Menu bar which is called as Header Navigation Bar (where Home  / Favorites / Catalog and all such custom link exists) . This can be done by changing CSS for classes like .HeaderTitleBarCell, .HeaderNavBarCell and .HeaderMenuBarText span element which is part of common.css file. This is already explained in lot of blogs and easy to do a one custom change to reflect everywhere.

Also we know, using customlinks.xml ([MiddlewareHome]/instances/instance1/bifoundation/OracleBIPresentationServicesComponent/coreapplication_obips1/customlinks.xml ) we can add multiple links in navbar and this is easy too.

Now , what if requirement ask you to change Header Nav bar link appearance differently based on different OBIEE Page or Dashboard you visit ? And what if you want to vanish certain links for specific set of users when they visit specific dashboard pages ?

Here is the solution using simple Javascript:

So here is my existing links:

And I want to Change the name from Catalog to something else only in specific dashboard page visit .

And it should be looks like below :

So I have changed font style , color and weight here . To do this add a Text section in corresponding OBIEE page and check HTML Markup and add below code . Also dont forget to add ‘display:none;’ is section properties css section otherwise you might get the expand/collapsible tiny icon.

<!–script>
var spanElement = document.querySelectorAll(“.HeaderMenuNavBarText span”)[4];
spanElement .innerHTML = ‘<!–font color=”#000000″ face =”Lato” size=”2″ ><!–b> CatalogMenuCustom</b–></font–>’ ;
</script–>

Remove the !- – and – – from all tags above. Also you can use any HTML code here and people might prefer to use the style tag as font tag is deprecated.

Now what I am doing here is searching the corresponding class for Nav Bar menu and looking for span entry 4 (item name :Catalog) which is as per sequence how it appears and overriding the text from ‘Catalog’ to ‘CatalogMenuCustom’

Similarly if you want to remove a certain menu item just find the sequence and use below code instead :

spanElement.innerText = ”;

This will make it a null item there which will make the menu item vanish but bear in mind that it might leave a small tiny space which is still clickable if somebody found it 🙂

This is 11g example but the principle should be same while doing customization in 12c.

 

 

Deploying new Font in OBIEE

Purpose: I want to use some font beyond the list of fonts available to use in OBIEE reports. OBIEE report has basic fonts(Arial , Courier, Garamond, Times , Tahoma) as below but I want new font list like below “Lato” fonts highlighted in yellow.

Lato is the the font designed by polish designer Lukasz. https://fonts.google.com/specimen/Lato

Here are the steps how to make this font available to use in OBIEE:

1. Download all the fonts (.ttf) files
2. Make “Lato” available to Operating system with 3,4 & 5 below
3. Deploy the lato font in OBIEE server nodes. In my linux environment, this has been installed under orabi user account who is the owner of OBIEE setup and installation. This should be fine if the Linux Admin install it using root .
4. First create a directory as ‘.fonts’ under your linux home dir , in my case it is ‘/home/orabi/.fonts’ and copied all .ttf files here. Then use command ‘fc-cache -fv’ in shell . This should show as below:

   

   Flush the OS font cache to load new 10 Lato fonts

5. Now deploy Lato font in OBIEE . Use below steps:
6. Edit [MiddlewareHOME]/Oracle_BI1/bifoundation/web/msgdb/messages/commonuitemplates.xml and Add Lato under : WebMessage name=”kuiFormatUIFontOptions” as below in Magenta:
   

   <WebMessage name=”kuiFormatUIFontOptions” translate=”false”><HTML><option value=”Arial”>Arial</option><option value=”Arial Black”>Arial Black</option><option value=”Arial Narrow”>Arial Narrow</option><option value=”Lato”>Lato</option><option value=”Lato-Black”>Lato-Black</option><option value=”Lato-BlackItalic”>Lato-BlackItalic</option><option value=”Lato-Bold”>Lato-Bold</option><option value=”Lato-BoldItalic”>Lato-BoldItalic</option><option value=”Lato-Hairline”>Lato-Hairline</option><option value=”Lato-HairlineItalic”>Lato-HairlineItalic</option><option value=”Lato-Italic”>Lato-Italic</option><option value=”Lato-Light”>Lato-Light</option><option value=”Lato-LightItalic”>Lato-LightItalic</option><option value=”Courier New”>Courier New</option><option value=”Garamond”>Garamond</option><option value=”Lucida Sans Unicode”>Lucida Sans Unicode</option><option value=”Microsoft Sans Serif”>Microsoft Sans Serif</option><option value=”Times New Roman”>Times New Roman</option><option value=”Tahoma”>Tahoma</option><option value=”Verdana”>Verdana</option></HTML></WebMessage>

7. Copy: [MiddlewareHOME]/Oracle_BI1/bifoundation/web/msgdb/messages/commonuitemplates.xml to [MiddlewareHOME]/instances/instance1/bifoundation/OracleBIPresentationServicesComponent/coreapplication_obips1/msgdb/messages/commonuitemplates.xml , if the directory doesn’t exist create it
8.  Now we need to update OBIEE schema directory to register this new information to OBIEE server. So edit [MiddlewareHOME]/Oracle_BI1/bifoundation/web/schemas/analysis_formats.xsd and add below in Magenta. Note the order matters as in which order you want the font list to be appeared in your UI display.
   

   <xs:simpleType name=”fontFamily”> <xs:annotation> <xs:documentation> family of fonts supported </xs:documentation> </xs:annotation> <xs:restriction base=”xs:string”> <xs:enumeration value=”Arial”/> <xs:enumeration value=”Arial Black”/> <xs:enumeration value=”Arial Narrow”/> <xs:enumeration value=”Lato”/> <xs:enumeration value=”Lato-Black”/> <xs:enumeration value=”Lato-BlackItalic”/> <xs:enumeration value=”Lato-Bold”/> <xs:enumeration value=”Lato-BoldItalic”/> <xs:enumeration value=”Lato-Hairline”/> <xs:enumeration value=”Lato-HairlineItalic”/> <xs:enumeration value=”Lato-Italic”/> <xs:enumeration value=”Lato-Light”/> <xs:enumeration value=”Lato-LightItalic”/> <xs:enumeration value=”Courier New”/> <xs:enumeration value=”Garamond”/> <xs:enumeration value=”Lucida Sans Unicode”/> <xs:enumeration value=”Microsoft Sans Serif”/> <xs:enumeration value=”Times New Roman”/> <xs:enumeration value=”Tahoma”/> <xs:enumeration value=”Verdana”/> </xs:restriction> </xs:simpleType>

9. After this bounce OBIEE . Simple opmn restart should work else bounce complete weblogic.
10. Now you can continue using Lato font everywhere in your Dashboard Page/Reports.

Note that : Be cautious once editing the analysis_formats.xsd and any typo here can cause the entire reports / catalog validation failed throwing “Catalog Object Validation Failed” or “Bad XML” . You can always spot this error and reason of objectionable tags under OBIPS logs . Check the log and it should tell what XML tag has issue.