Building Scalable Systems - Handling Hyper-Growth

Hyper-growth is one of the most exhilarating phases for any business, marked by rapid expansion, new opportunities, and an undeniable sense of momentum. Yet, alongside this excitement lies a unique set of challenges.

I’ve seen businesses stumble not because of a lack of ambition or potential but due to difficulties in planning effectively for future needs. Is this because long-term planning is inherently complicated? Absolutely not.

The Phoenix Project - My Thoughts

This book has been incredibly insightful and has been on my reading list for quite some time. The story resonates deeply with me, especially in how it connects to my work. It offered me a fresh perspective and a chance to reflect. While some principles might feel familiar or even a bit cliché, it's always valuable to revisit and be reminded of best practices.

Here are some key takeaways and reflections from the book that I’d like to share.

Embrace Systems Thinking: Optimize the Entire System

Main Security Tables Cheat Sheet

ACCESS PROFILES
PSACCESSPRFL

USERS
PSOPRDEFN
PSOPRDEFN_LANG
PSOPRALIAS
PSROLEUSER
PSUSERATTR
PSUSEREMAIL
PSUSERPRSNLOPTN
PS_ROLEXLATOPR
PS_RTE_CNTL_RUSER

ROLES
PSROLEDEFN
PSROLEDEFNLANG
PSROLECANGRANT
PSROLECLASS

PERMISSION LISTS
PSCLASSDEFN
PSCLASSDEFN_LNG
PSAUTHSIGNON
PSAUTHITEM
PSAUTHPRCS
PSAUTHCUBE
PSAUTHMP
PSAUTHBUSCOMP
PSAUTHQUEUEMON
PSPRCSPRFL
PS_SCRTY_QUERY
PS_SCRTY_ACC_GRP
PS_SCRTY_ADS_AGRP
PSAUTHOPTN
PSAUTHWS
PS_SCRTY_ADS
PS_SCRTY_SRCHGRP
PS_MC_OPR_SECURITY
PS_MC_OPRID

DEFINITION SECURITY
PSPTDEFSEC_GRPS
PSPTDEFSEC_GRP
PSPTDEFSECINRL
PS_APP_DES_OBJ_CST
PSOPROBJ

PERSONALIZATIONS
PSUSEROPTNDEFN
PSUSEROPTNLANG
PSOPTNCATGRPLNG
PSOPTNCATGRPTBL
PSOPTNCATTBL
PSOPTNCATLANG

SECURITY OPTIONS
PSSECOPTIONS

SECURITY LINKS
PSUSEROTHER
PSUSEROTHER_L
PSUSERSELFOTHER
PSUSERSELFOTH_L
PSROLEOTHER
PSROLEOTHER_LNG
PSPERMLISTOTHER
PSPERMLISTOTH_L

USER ID TYPES
PSOPRALIASTYPE
PSOPRALIASFIELD

USER BYPASS TABLE
PS_BYPASS_TABLE

FORGOT EMAIL TEXT
PSPSWDEMAIL
PSPSWDEMAILLANG

PASSWORD HINTS
PSPSWDHINT
PSPSWDHINT_LANG

SIGNON PEOPLECODE
PSSIGNONPPC

DIRECTORY
PSDSDIR
PSDSSRVR
DSCONNECTID
PSDSEXT_INSTALL
PSDSSECMAPMAIN
PSDSSECMAPSRVR
DSUSRPRFLMAP
PSDSUSERPRFL
PSDSSECROLERULE
DSSRCH_SBR
DSSRCHATTR
DSSECFILTER
PT_WF_NOT_DSCFG

SignOn PeopleCode

Consider there is a requirement to warn the users while they login and before they land on the peoplesoft home page. How can we achieve this?

When this question came to me I was not able to understand how we can recognize the user who logs in through the login page. The user can be an employee, he can be a guest. But before landing the homepage how is it possible to check the role attached to the user? That was a challenge indeed at first.

SOPC - Sign on People code was one way for implementing the same. This people code will get fired upon the login to any PIA environment.

How to relate the people code we write to the login session. This can be achieved through: PeopleTools >> Security >> Security Objects >> Sign on people code

You can provide the record name, the field and the event on which the people code is written. Then use the checkbox to control when the code need to be fired.

Now consider, you need to show the user a page based on how he is verified. For this, you have to use the page under: Web profile >> Web profile configuration >> Look & Feel

Use the Password section to cater the above requirement. In the password warning page you can provide your own HTML page which you have created with the necessary warning for the user.

And use the SetAuthenticationResult function to do the rest of the work.

Security Processes

Refresh SJT_OPR_CLS and Refresh SJT_CLASS_ALL are Security process.

When to execute Refresh SJT_OPR_CLS process

On the Roles - Permission Lists page

• Add a permission list with data permission to a role that is already assigned to one or more users.

• Remove a permission list with data permission from a role that is already assigned to one or more users.

On the User Profile component

Add a row security permission list.

• Delete a row security permission list.

• Add a role with data permission.

• Delete a role with data permission.

• Clone an existing profile that has data permission through roles or a row security permission list.

•Deactivate a user.

You can refresh SJT_OPR_CLS in real-time by using the subscriptions on the USER_PROFILE and ROLE_MAINT messages or on demand using the SCRTY_OPRCLS app

engine process.

Run the Refresh SJT_CLASS_ALL process when

• Add a new department to a department security tree.

• Delete a department from a department security tree.

• Move a department to another parent node in a department security tree.

• Modify a department security tree.

Understanding Row Level Security

I went through the post below from Grey Sparling Solutions Blog. This post explains the concept of row level security in a very understandable way. Go through the same.

Lot of the application groups within PeopleSoft provided their own row level security setup - HR with the security based on the department tree being a great example. The Financials group went beyond just one type of row level security "out of the box" - I forget how many choices they offered - there were several different common chart of account fields (business unit, department, account, etc.). With the application teams providing row level security in the vanilla install, many customers didn't realize that it is possible to handle row level security differently.

One thing that I always found helpful in customer meetings when this topic came up was to go through exactly how the PeopleTools component processor used search records to handle security. Once you understand that, then you can best decide if changing the row level security that the application groups provide makes sense from a cost/benefit perspective. And there definitely is a cost in the current PeopleTools - fixing row level security so that it was brain dead simple for customers to pick and choose how they wanted to implement it was one of the big ticket items planned for PeopleTools 9.

First, a quick side journey into the component processor (catch me at a conference or user group meeting sometime and I'll tell you the inside story on where the "component" name came from - it'll make you laugh). The component processor is truly the guts of PeopleTools - it is what handles the business logic, database commits, etc. for just about every page that you see in a PeopleSoft application. PeopleSoft components are parent/child hierarchies of database records that automatically get pulled together as one unit of work from the application developer's perspective.

There is one row of data at what is called Level 0. There can be up to 3 levels beyond that. Each level is required to have the same unique keys as the level above it, plus at least one additional unique key field to identify what makes the rows in that level unique.

For example, if a customer master record is at level 0 and has CUSTOMER_ID as it's unique key, you might have customer contacts at level 1 keyed by CUSTOMER_ID and CONTACT_ID. Customer contact phone numbers might be at level 2, keyed by CUSTOMER_ID, CONTACT_ID and PHONE_ID.

The data at each level comes from the key values above it. If the CUSTOMER_ID at level 0 is 1234, then the component processor would automatically use 1234 for CUSTOMER_ID for selecting data into level 1, and so on, down to level 3 (there are ways to override this, but that's how the default behaviour works).

The important thing to keep in mind here from the perspective of row level security is that populating the key fields in the level 0 record drives everything else. This is what the seach dialogs are responsible for.

When you first enter a component via the browser you get a page asking you for whatever the key fields are. In this case it would just be CUSTOMER_ID, but if the level 0 record has multiple key fields, then you'd get prompted for those as well. If you don't know the values, you click Search and get presented with a list of valid values.

But where does that list come from? Each component has what is known as a search record. The component processor uses the search record to supply the values for the key fields of the level 0 record, which then flow down to the child levels.

Any rows of data that the search record returns to the user gives the user access to that data.

So if you wanted to limit someone to only look at active customers, then you could create a view that had the same key fields as the customer master record, but had a WHERE clause like " WHERE CUSTOMER_STATUS='A' ". The view would only return active customers, which means that no one could get into that component to look at an inactive customer.

The component processor recognizes a couple of "special" fields on search records - OPRID and OPRCLASS. If the field OPRID exists on the search record, it automatically gets filled in with the current logged in user's ID. If the field OPRCLASS exists on the search record, it gets filled in with the Row Level Security Class (Permission List) that is specified on the user's security profile.

And that's essentially it for how the component processor handles things for row level security. There are a couple of PeopleCode events that fire when a search dialog is initialized (SearchInit) and when the user clicks "Search" (SearchSave), but those are not really intended for row level security. I'll write more about that in the future.

So how does the department tree security in HR work then? The HR group created a setup page that lets you pick a security class (permission list) and a tree node from the department security tree. For each one of these combos that you select, you decide whether access is granted or denied. All of this data is stored in a table. The HR team then created a view that links this table, the tree manager tables, and the employee department data together. The component processor automatically plugs in the row level security class for the user when selecting from the view, which limits what the user can see.

Financials works in a similar fashion, although the performance of the trees in the view didn't work well enough for them so they ended up with flatter security structures. The HR team actually delivered a couple of de-normalizing utilities for larger customers to deal with this issue as well.

So, how do you change that delivered security?

You could clone the delivered security views (and the tables/pages that they use to maintain the data). I helped a customer once do this where they wanted to use the position tree instead of the department tree.

You can also create your own security structures that do not mimic the delivered ones. All that matters is that you have a table or view that will return the key values that a user has access to. All that matters is that the search record has the same key fields as the level 0 record in the component that you're securing and that you either have the OPRID or OPRCLASS there to provide security.

There are no restrictions about what data the search record looks at or how that data gets maintained. For example, you might have some batch job that updates the table based on security defined in some totally different system. Or maybe someone has to request access to look at some data and security administrator will update the table that the search record looks at. I know of one customer that wanted to have their call center employees only be able to look at customer data when a call came in. So right before the server would send the screen pop for the customer data down to the agent's desktop, the security tables would be updated for that user to have access. As soon as the call was over, then access was taken away.

So the development of a new search record/view is one cost. This includes not just the technical development, but the auditing to make sure that it works properly as well.

Another cost is that the search record is attached to the component definition. So if you update a whole bunch of components to point to different search records, then you have to keep track of that at upgrade time. This is less costly than doing things like changing delivered code since you just need to add one extra step in your upgrade - update any changed components again. Not really a lot of thinking/analysis involved. But, a lot of customers like to run as close to vanilla as they can, so it is worth keeping in mind.

Of course, you don't want to be so afraid of a small cost at upgrade time if your business requirements need different data level security. I talked with a customer once that had turned off row level security in their Purchasing implementation because the delivered methods didn't work for them. Their auditors were complaining (and this was even before all of the current SarbOx push) and they wanted to know when Purchasing would support what they wanted. Changing the row level security is not so hard/costly that you'd want to run without security.

 

Security Views

The security views work by joining together the security join tables along with the OPRID of the user accessing the view. All security views share some characteristics both in their structure and their view text.

Each view must include the OPRID as a key field, the keys for the transaction data, and any additional fields that are used as search or list box fields. Most core views also include other fields from the security join record that other views can enter where criteria against.

We are not including ROWSECCLASS as a key in the security views. This is because if we do, then we would have to require a ROWSECCLASS be entered for every OPRID. The reason for this is that when the join to PSOPRDEFN is done behind the scenes, it will put the value of PSOPRCLS into the field returned from PSOPRDEFN.ROWSECCLASS if the ROWSECCLASS is blank. If the PSOPRCLS is also blank, then the OPRID is returned. This makes it impossible to join to the SJT_CLASS_ALL table. Including the ROWSECCLASS as a non-key field in the security view is okay, but does not provide any benefit. You will still find it in some views because it was less risky to leave it there for now.

The core views need to join in PSOPRDEFN, SJT_CLASS_ALL, and the appropriate transaction SJT. For example, for People with Jobs and/or People without Jobs the SJT record is SJT_PERSON.

About Security Processes

Refresh SJT_OPR_CLS and Refresh SJT_CLASS_ALL are Security process.

When to execute Refresh SJT_OPR_CLS process

On the Roles - Permission Lists page

• Add a permission list with data permission to a role that is already assigned to one or more users.
• Remove a permission list with data permission from a role that is already assigned to one or more users.

On the User Profile component

Add a row security permission list.
• Delete a row security permission list.
• Add a role with data permission.
• Delete a role with data permission.
• Clone an existing profile that has data permission through roles or a row security permission list.
• Deactivate a user.

Note: You can refresh SJT_OPR_CLS in real-time by using the subscriptions on the USER_PROFILE and ROLE_MAINT messages or on demand using the SCRTY_OPRCLS appengines process.

Run the Refresh SJT_CLASS_ALL process when

• Add a new department to a department security tree.
• Delete a department from a department security tree.
• Move a department to another parent node in a department security tree.
• Modify a department security tree.