Tuesday, May 27, 2014

OpenIDM 3.0 . July 2014 . Product Enhancements

It has been confirmed that OpenIDM 3.0 will be released in July 2014. The new release is now undergoing QA - should be ready soon. ( Ok, usually QA will take a while... :> )

I'm still attending the webinar by Anders - "OpenIDM: Identity Administration for Users, Devices and Things".

Quick summary of some of the product enhancements to be expected from this new release.



Of course, the key highlight of this release is Role-Based Provisioning support.


.



Wednesday, May 14, 2014

OpenAM Session Persistence and Cross-talk



Something is brewing in the future release of OpenAM regarding the Session Persistency and Cross-talk among OpenAM servers.





I am seeing the following in the documentation:



To enable crosstalk between OpenAM instances, click the Enabled box. This ensures that OpenAM instances talk with each other to resolve any non-local sessions, rather than using session persistence. 
Note that if you have a large numbers of requests (for example, millions of requests per hour), the crosstalk could adversely affect server performance with the increase in network traffic. 
If session persistence is enabled, the crosstalk option is not enabled by default. If session persistence is not enabled, crosstalk is automatically enabled by default. If both session persistence and crosstalk is enabled, both features will be implemented, which could result in more network traffic.
 

I'm looking forward to this new enhancement!
 

.


Sunday, May 11, 2014

amUpgrade log files issue

I was trying out upgrading our test labs' OpenAM server from 10.1.0-Express to 11.0.1 and encountered 100% full on our file-system.


How can that be? Our test VM has 26G of space allocated.

So I started investigating what was the root cause. Found a huge log file that is 3.3G in size - amUpgrade.



I then executed a tail command on the file. Wow! It was moving every single milli-second.




Just found out this was reported in bugster OPENAM-1791 - When in message level debug, amUpgrade file logs entries on every request to OpenAM console.

Time to patch your OpenAM if your are on the affected version. Workaround is not to turn on MESSAGE debug level, unless required.





.


Saturday, May 10, 2014

OpenAM 11.0.1 Upgrade - Fail

I had some free time the day before yesterday and so went ahead to play around with OpenAM 11.0.1 upgrade. My test environment was 10.1.0-Express.



Not so smooth initially. Hit into "Connect Error: No operational connection factories available" error during upgrade.





amUpgrade:05/08/2014 05:53:53:317 PM SGT: Thread[ajp-apr-192.168.0.88-8009-exec-1,5,main]
ERROR: Error occured while upgrading OpenAM
org.forgerock.openam.upgrade.UpgradeException: Connect Error: No operational connection factories available
at org.forgerock.openam.upgrade.steps.UpgradeEntitlementsStep.perform(UpgradeEntitlementsStep.java:174)
at org.forgerock.openam.upgrade.UpgradeServices.upgrade(UpgradeServices.java:172)
at com.sun.identity.config.upgrade.Upgrade.doUpgrade(Upgrade.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.click.util.ClickUtils.invokeMethod(ClickUtils.java:3317)
at org.apache.click.util.ClickUtils.invokeListener(ClickUtils.java:2088)
at org.apache.click.control.AbstractControl$1.onAction(AbstractControl.java:228)
at org.apache.click.ActionEventDispatcher.fireActionEvent(ActionEventDispatcher.java:259)
at org.apache.click.ActionEventDispatcher.fireActionEvents(ActionEventDispatcher.java:236)
at org.apache.click.ActionEventDispatcher.fireActionEvents(ActionEventDispatcher.java:180)
at org.apache.click.ClickServlet.performOnProcess(ClickServlet.java:746)
at org.apache.click.ClickServlet.processAjaxPageEvents(ClickServlet.java:1860)
at org.apache.click.ClickServlet.processPage(ClickServlet.java:559)
at org.apache.click.ClickServlet.handleRequest(ClickServlet.java:383)
at org.apache.click.ClickServlet.doGet(ClickServlet.java:276)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.forgerock.openam.validation.ResponseValidationFilter.doFilter(ResponseValidationFilter.java:44)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at com.sun.identity.setup.AMSetupFilter.doFilter(AMSetupFilter.java:113)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:197)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1822)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.forgerock.openam.upgrade.UpgradeException: Connect Error: No operational connection factories available
at org.forgerock.openam.upgrade.steps.UpgradeEntitlementsStep.upgradeEntitlementIndexes(UpgradeEntitlementsStep.java:338)
at org.forgerock.openam.upgrade.steps.UpgradeEntitlementsStep.perform(UpgradeEntitlementsStep.java:170)
... 41 more



I thought the embedded OpenDJ was dead, but upon checking, it was running as per normal after it was upgraded from version 2.4.6.8102 to 2.6.1.10289.


ERROR: Upgrade required: upgrading from 8102 to 10289

>>>> OpenDJ Upgrade Utility

 * OpenDJ will be upgraded from version 2.4.6.8102 to 2.6.1.10289
 * See '/home/azlabs/var/openam1/opends/upgrade.log' for a detailed log of
 this operation

>>>> Preparing to upgrade

  OpenDJ 2.5.0 modified the default configuration of the 'isMemberOf' virtual
  attribute so that it is included with group entries. This was done in order
  to make it easier for users to determine which groups a 'nested' group
  belongs to.
  Do you want to make this configuration change? (yes/no) yes

  The upgrade is ready to proceed. Do you wish to continue? (yes/no) yes

:
:

>>>> Performing upgrade
>>>> OpenDJ was successfully upgraded from version 2.4.6.8102 to 2.6.1.10289

 * See '/home/azlabs/var/openam1/opends/upgrade.log' for a detailed log of this operation


No idea then. So what to do? Clicked on Upgrade to OpenAM 11.0.1 hyperlink again. Ops! No good. 





What to do next? Re-do! It's quite simple as I have already backed up every directories.

1. Stop application server
2. Remove the "corrupted" var directory
3. Untar the backed up var directory
4. Start application server again

This time round, everything went through nicely.

So why?


I suspect my VM was too slow (The VM was a shared resource with other projects running). See the broken image which the red arrow is pointing to below?







.


Friday, May 9, 2014

OpenAM Backup and Restoration




There's a discussion in OpenAM mailing list regarding export-svc-cfg sub-command. This command is related to Backing Up and Restoring OpenAM Configurations in OpenAM Administration Guide.





My friend, Peter, has something to say (he usually does. Ha! *joking*) :

export-svc-cfg and import-svc-cfg aren't necessarily the most trustworthy backup solution for configuration data. An example where things can go wrong is OPENAM-718 and OPENAM-3793. Since my recent encounters with these commands personally I'm more inclined to use OpenDJ's export-ldif command for backup.





This email thread brought me back to an discussion I had with a customer a month back. He was reading the same document from ForgeRock and wanted to implement the same for his backup and restoration procedure.

That's not necessary, I told him. When we implemented the project, we already have a Backup and Restoration procedure in place.

Maybe Azimuth's way of Backup and Restoration procedure looks too old-school, not sexy enough.

Nightly Backup
1. Backup OpenAM war directory
2. Backup OpenAM var directory (this would already back-up the configuration data in the embedded OpenDJ)
3. Backup OpenDJ data in LDIF format and DB format (for customers with external OpenDJ)

Restore
1. Restore OpenDJ data (for customers with external OpenDJ)
2. Restore OpenAM var directory
3. Restore OpenAM war directory


The beauty/concept of embedded OpenDJ has to be appreciated. It's embedded and thus backing up the var directory will be more than sufficient. That's my own opinion.

I'm not here to argue about which approach is better. I'm more of a practical person when it comes to operation. It has to be fast for roll-back/restoration with minimal downtime as much as possible.


.


Thursday, May 8, 2014

Profile Attributes Processing - Map Key

I have a customer who asked me whether the same Map Key can be mapped to multiple Corresponding Map Value. This is regarding Profile Attribute Processing for Policy Agent configuration.




He wants to map UID from data source to USERNAME and EMPLOYMENTID. This is part of his application team's requirement during Policy Agent integration.




I tested in our test environment. Not possible.






The OpenAM debug log will throw the following stack trace:

ERROR: AgentsRepo.setAttributes(): Unable to set agent attributes
Message:Data validation failed for the attribute, com.sun.identity.agents.config.profile.attribute.mapping

at com.sun.identity.sm.ServiceSchemaImpl.throwInvalidAttributeValuesException(ServiceSchemaImpl.java:647)
at com.sun.identity.sm.ServiceSchemaImpl.validatePlugin(ServiceSchemaImpl.java:628)
at com.sun.identity.sm.ServiceSchemaImpl.serverEndAttrValidation(ServiceSchemaImpl.java:599)
at com.sun.identity.sm.ServiceSchemaImpl.validatePlugin(ServiceSchemaImpl.java:557)
at com.sun.identity.sm.ServiceSchemaImpl.validateAttrValues(ServiceSchemaImpl.java:512)
at com.sun.identity.sm.ServiceSchemaImpl.validateAttributes(ServiceSchemaImpl.java:289)
at com.sun.identity.sm.ServiceConfig.setAttributes(ServiceConfig.java:536)
at com.sun.identity.idm.plugins.internal.AgentsRepo.setAttributes(AgentsRepo.java:1049)
at com.sun.identity.idm.server.IdServicesImpl.setAttributes(IdServicesImpl.java:1701)
at com.sun.identity.idm.server.IdCachedServicesImpl.setAttributes(IdCachedServicesImpl.java:528)
at com.sun.identity.idm.AMIdentity.store(AMIdentity.java:589)
at com.sun.identity.console.agentconfig.model.AgentsModelImpl.setAttributeValues(AgentsModelImpl.java:790)

:
:
amIdm:05/07/2014 05:08:17:912 PM SGT: Thread[ajp-apr-192.168.0.88-8009-exec-4000,5,main]
WARNING: IdServicesImpl.setAttributes: Unable to modify identity in the following repository com.sun.identity.idm.plugins.internal.AgentsRepo :: Plug-in com.sun.identity.idm.plugins.internal.AgentsRepo: Error while setting attributes for agentonly=iticket
amIdm:05/07/2014 05:08:17:912 PM SGT: Thread[ajp-apr-192.168.0.88-8009-exec-4000,5,main]
WARNING: IdServicesImpl.setAttributes: Unable to set attributes  for identity agentonly::iticket in any configured data store
Message:Plug-in com.sun.identity.idm.plugins.internal.AgentsRepo: Error while setting attributes for agentonly=iticket

at com.sun.identity.idm.plugins.internal.AgentsRepo.setAttributes(AgentsRepo.java:1060)
at com.sun.identity.idm.server.IdServicesImpl.setAttributes(IdServicesImpl.java:1701)
at com.sun.identity.idm.server.IdCachedServicesImpl.setAttributes(IdCachedServicesImpl.java:528)
at com.sun.identity.idm.AMIdentity.store(AMIdentity.java:589)
at com.sun.identity.console.agentconfig.model.AgentsModelImpl.setAttributeValues(AgentsModelImpl.java:790)
at com.sun.identity.console.agentconfig.AgentProfileViewBean.handleButton1Request(AgentProfileViewBean.java:395)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)



I checked the attribute in the embedded configuration store (OpenDJ). It should be able to handle same Map Key be mapped to multiple Corresponding Map Value scenario.


There is no error found in OpenDJ access log as well.

So this must be a feature. I suppose. 


.



Monday, May 5, 2014

OpenAM Policy Agent 3.3.0 - Extended URL validation


In the latest release of OpenAM Policy Agent 3.3.0, some behaviours have been changed - see Important Changes to Web Policy Agent Functionality.



One of the changes is the way how Naming URL validation works.

Naming URL validation was introduced after release 3.0.4. The initial implementation of naming URL validation for web policy agents enabled validation by default. Naming URL validation is now fully disabled by default. You can adjust this setting by using the bootstrap configuration property, com.forgerock.agents.ext.url.validation.level.


# Extended URL validation
#
# - level:              0 - extended URL validation;
#                          1 - simple URL validation;
#                          2 - (default) validation disabled.

com.forgerock.agents.ext.url.validation.level = 2



The default value in Policy Agent 3.3.0 onwards is 2, which means validation is disabled. Is this good? Well, it's debatable.


Below is what will be shown upon Policy Agent starts up:


2014-04-30 10:20:57.887       -1 12817:d7b0450 all: =======================================
2014-04-30 10:20:57.887       -1 12817:d7b0450 all: Version: 3.3.0
2014-04-30 10:20:57.887       -1 12817:d7b0450 all: 
2014-04-30 10:20:57.887       -1 12817:d7b0450 all: Build Date: Nov  8 2013 21:58:51
2014-04-30 10:20:57.887       -1 12817:d7b0450 all: Build Machine: constable.internal.forgerock.com
2014-04-30 10:20:57.887       -1 12817:d7b0450 all: =======================================
2014-04-30 10:20:58.072       -1 12817:d7b0450 all: naming_validator(): validation disabled


Personally, I do not like it, especially if this is a new agent installation.

If com.forgerock.agents.ext.url.validation.level is set to 1, simple URL validation will be enabled. The logs is showing more information.


2014-04-30 10:57:32.869       -1 26488:151d5470 all: =======================================
2014-04-30 10:57:32.869       -1 26488:151d5470 all: Version: 3.3.0
2014-04-30 10:57:32.869       -1 26488:151d5470 all: 
2014-04-30 10:57:32.869       -1 26488:151d5470 all: Build Date: Nov  8 2013 21:58:51
2014-04-30 10:57:32.869       -1 26488:151d5470 all: Build Machine: constable.internal.forgerock.com
2014-04-30 10:57:32.869       -1 26488:151d5470 all: =======================================
2014-04-30 10:57:32.883       -1 26488:151ee220 all: naming_validator(): failed to read current index value, defaulting to https://xxo-uat.true.th:11443/openam/namingservice
2014-04-30 10:57:32.884       -1 26488:151ee220 all: naming_validator(): still staying with https://www-uat.xxx.com:11443/openam/namingservice
2014-04-30 10:57:33.935       -1 26488:151ee8c0 all: url_validator(0): https://www-uat.xxx.com:11443/openam/namingservice validation succeeded




But .... that's still not enough to verify whether the agent has been installed correctly. If the agent profile name and/or password are set up wrongly, you'll still be getting url_validator(0): https://www-uat.xxx.com:11443/openam/namingservice validation succeeded.

Try setting com.forgerock.agents.ext.url.validation.level to 0, extended URL validation will be enabled. The logs will display even more information.


2014-04-30 10:58:41.831       -1 26617:1a923420 all: =======================================
2014-04-30 10:58:41.831       -1 26617:1a923420 all: Version: 3.3.0
2014-04-30 10:58:41.831       -1 26617:1a923420 all: 
2014-04-30 10:58:41.831       -1 26617:1a923420 all: Build Date: Nov  8 2013 21:58:51
2014-04-30 10:58:41.831       -1 26617:1a923420 all: Build Machine: constable.internal.forgerock.com
2014-04-30 10:58:41.831       -1 26617:1a923420 all: =======================================
2014-04-30 10:58:41.846       -1 26617:1a93c210 all: naming_validator(): failed to read current index value, defaulting to https://www-uat.xxx.com:11443/openam/namingservice
2014-04-30 10:58:41.846       -1 26617:1a93c210 all: naming_validator(): still staying with https://www-uat.xxx.com:11443/openam/namingservice
2014-04-30 10:58:43.015       -1 26617:1a93cae0 all: NamingValidateHttpLogin() response:



2014-04-30 10:58:43.015       -1 26617:1a93cae0 all: url_validator(0): https://www-uat.xxx.com:11443/openam/namingservice validation failed with OpenAM authentication service failure (3), http status (200)


Now that the agent is verified to be installed correctly, switch com.forgerock.agents.ext.url.validation.level to 2 and we are ready to go!



.

Saturday, May 3, 2014

Open-source software projects handling of security flaws

I blogged about Open-source software projects need to improve vulnerability handling practices some time ago.

This morning, I received an email from Zimbra - Zimbra News: Important Zimbra Collaboration Security Updates.




One thing to note is we are not a paid customer of Zimbra, but we have been running Zimbra Community Free Edition for a long time. Yes, not a single cent for Zimbra to earn. 

But, hey ... they still send us emails regarding security updates... as and when they are found over the years!


At the same time, Zimbra Collaboration 7.2.7 has been released, which fixes some security issues as well.



In the release note, there is a dedicated section on Security Fixes, where the security flaws are described and fixes/workaround are recommended.





Cool, right? Still remember we are not a paid customer?



Now, we know there is a commotion lately regarding the way ForgeRock handles the latest security flaws found in OpenAM.


In late 2012 (well, when ForgeRock was still young), they published security advisory publicly to the community. It is still available in the Internet today.

The root causes and workarounds were publicly shared then.




Moving forward, when the latest security flaws are discovered, ForgeRock now decides to only briefly inform the community with the following email in the mailing list.

Dear OpenAM Community Member, 
ForgeRock today issued security fixes for OpenAM releases 9.5.x, 10.0.x and 11.0. 
We are sending this notice to strongly recommend that community members update their OpenAM production deployments at the earliest opportunity. 
For OpenAM 11.0, fixes have been applied to trunk. If you are a customer, then you can upgrade to 11.0.1, which contains the fixes, and is available from the ForgeRock Customer Portal. 
For OpenAM 9.5.x and 10.0.x deployments, contact us at http://go.forgerock.com/2014OpenAM1101SecurityAlert.html 
Regards,

Security advisory is now no longer publicly available. It's now a simple one-liner: ForgeRock today issued security fixes for OpenAM releases 9.5.x, 10.0.x and 11.0.

Now, when I posted OpenAM 11.0.1 and Policy Agent 3.3.1 released!, I was of the impression that the difference between paid customers and non-paying community members is that a JAR is easily made available to the former.

If you are a paid customer and on the latest release of OpenAM, the patch is fairly simple.
1. Download the appropriate jar for your OpenAM version 
2. Restart your application server

Done. As simple as that. 

And I openly and still honestly think that this "convenient service" is what paid customers deserve. (Come on, there has to be a difference. Otherwise, who will pay premium service if everyone enjoy the same benefits?) On the same day, I tweeted a sentence from OpenAM mailing list - "Open Source software is NOT free. Open Source software still takes a lot of time and money in order to produce it." I was still on the same context on the "convenient service" provided by ForgeRock to paid customers.

What I did not know then (which someone later pointed out to me. Thank you very much!) was the detailed descriptions and codes that have been fixed are only made known to paid customers. The community members are expected to look at the source code trunk and to compare what has been changed. Even if the fixed codes have been found, one has to guess what exactly was the root cause to the security flaw. But all these delays will take time.

I do not agree to this. Security flaw is big issue -- to paid customers … as well as non-paying community members. Every single deployed OpenAM should be patched at the earliest time possible.


Take a look at Redhat …



Everyone single piece of information is shared publicly, including "security".



Paid Redhat customers can immediately download the latest binaries to patch their servers, while community members need to quickly see what has been changed and to apply the same into their servers.



In my opinion, paid customers should be differentiated by:

  1. premium, top-notch & quick turnaround support services
  2. product features


Security fixes should not be classified as product features. It should never be.


You see.. the point is: Yes, I may not be a paid customer now. But if your product is great and your service is excellent and you treat security as your top priority, I may well pay you for commercial support one day. Better still... for my customers, I'll definitely push your paid services to them. 


Dun you get it? I thought this is Business 101?

.

Friday, May 2, 2014

OpenDJ - Internet Scale

I attended a webinar - Building Directory Services for IRM earlier this week. The presenter is non other than a good friend of mine - Ludovic, Product Manager for ForgeRock OpenDJ.

And I always like this slide from him. It's cool!




The "ferris-wheel" thingy is actually a visual representation of OpenDJ replication topology. It was generated by a script written by Chris Ridd last year - see OpenDJ: Visualizing the Replication Topology.

So, the above topology is configured as such:

1. 4 data-centers across the globe
2. 2 OpenDJ replication servers in each data-center
3. 12-15 OpenDJ directory servers connected to each OpenDJ replication server
4. Total of 200 OpenDJ directory servers deployed
5. 100+ millions entries! Wow!



By the way, this topology is from an actual customer deployment. Yes, it's LIVE in production.

Now, this is indeed Internet Scale.



.

Thursday, May 1, 2014

Account Lockout feature in OpenAM

As per documented, there are 2 ways for Account Lockout in OpenAM:

OpenAM supports two different approaches to account lockout, where OpenAM locks an account after repeated authentication failures. Lockout works with modules for which users can enter a password incorrectly. 
Memory lockout locks the user account, keeping track of the locked state only in memory, and then unlocking the account after a specified delay. Memory lockout is also released when OpenAM restarts. 
Persistent (physical) lockout sets the user account status to inactive in the user profile. For persistent lockout, OpenAM tracks failed authentication attempts by writing to the user repository. 
Persistent account lockout works independently of account lockout mechanisms in the underlying directory server that serves as the user data store.

I am in Bangkok this week and this is the 1st question (How to configure Account Lockout?) I get from customer when I arrived. I blogged on this topic before.

In particular, Customer only wants memory lockout. Why? I seldom have this requirement. Usually, persistent lockout is the default. Ok, in fact, I seldom have requirement to configure Account Lockout feature in OpenAM at all. My recommendation is to always ride on the backend authoritative user store's (e.g. OpenDJ, Microsoft Active Directory) Password Policy.

Anyway, back to Customer's question. Account Lockout can be easily turned on by clicking on the Enabled check-box next to "Login Failure Lockout Mode". It is disabled by default.





For memory lockout, remember to uncheck Enabled check-box next to "Store Invalid Attempts in Data Store".




Simple. Isn't it?

For persistent lockout, OpenAM sets the value of the user's inetuserstatus profile attribute to inactive. You can also specify another attribute to update on lockout. You can further set a non-default attribute on which to store the number of failed authentication attempts. When you do store the number of failed attempts in the data store, other OpenAM servers accessing the user data store can also see the number

To reverse the bold statement above: "When you do not store the number of failed attempts in the data store, other OpenAM servers cannot see the number."


Well, for multi-instance deployment, behaviour will be kind of erratic. Just take note.



.