Become a leader people love

1. LEADING WITH A HEAVY HEART

• Read the room, tap into pour humaneness, focus on what is possible.
• Carry on continue in business while still bearing the weight on what is happening to us, families, lives, and those we love, our companies, employees and our communities.
• Heart, feelings, empathy - contemporary leadership.
• Sometimes the right thing to do is not try to be happy.
• Saying let's try to be positive at times when things are clearly not okay can create gap between the leader and the team.
• Creates crisis of confidence - team may say we see what you cannot see.
• Emotional labour - is the gap between who we are naturally, and who we put effort into becoming for a situation. Bigger the gap more the stress and challenges.
• Instead of being trying to happy and staying positive, think of what is possible. 
• Focus on this -----
• Things might not be okay, but there are opportunites to move forward.
• Confront reality
• Ground yourselves on what is going on.
• What we think it means to us.
• And what might come next.
• Confronting reality might sound like this:
• Here is what we know.
• Here is what we d not know, but that we are working on.
• This is what we think it will mean.
• It is far more important to have the right questions than the right answers.
• Identify and focus on what can be controlled.
• Maybe there are areas where we may not have control but we may have influence.
• Discuss on sphere of activity we are leaning into. 
• Can i control the individual experiences of my team members.
• Can i influence other things they are experiencing in a positive way. Talking to them on that, agreeing on that, etc. etc.
• Communicate that there are certain things that are not in your control. 
• Be honest about things around you that are not in our control.
• Invite team members to share their perspective. Questions
• How are you doing? Ask for their perspective of things. 
• What's one thing I can do to be a better leader for you?
• If you ever left us, what would be the reason? How can we make sure that does not happen?
• When you are personally in deep pain
• Consider your Mental health and wellbeing
• Are you physically okay
• Are you mentally okay
• Are you emotionally okay
• Can you be in the game?
• Take time for yourself.

2. SERVANT LEADERSHIP

• Servant leadership is all about teams winning. When they win, you win.  
• Habits
• External self
• Reflective self
• Don't become a slave of rat race. Think reflectively. 
• How do you want to be remembered?
• What are your values?
• The number 1 customer for companies is its people.

3. JOB MOTIVATION

• Your underlying premise about others should be:
• Do care, they want to contribute, they want to do right thing and want to contribute. They ware willing to work hard.
• Above premise is more noble, and closer to human nature. 
• This will lead to success for both you as leader and teams as well.
• Build workplaces that go with the grain of human nature.
• Controlling Contingent Reward
• Also called IF-THEN Reward.
• If you do this, then you get that.
• Are extremely effective with simple tasks with short time horizons.
• Because people love rewards.
• If then rewards are less effective for complex tasks with longer time horizons.
• Above sentence is more relevant to white collar jobs.
• Money is a motivator. Money is proxy for fairness
• Internal fairness. You and i do same kind of job, contribution etc. If I know you are getting 10% more, then i get demotivated. Because we are doing same work, but you are getting more.
• You and I are getting same amount, but another Org in same labor market for same work is paying 20% more. We get demotivated. Because we think it is not fair.
• Humans are attuned to the norm of fairness.
• If you violate the notion of fairness, you are toast.

If you are running an organization or managing a team, then to motivate your team, put the two key ideas in place:

- Pay people fairly. Paying people failry does not mean paying everyone equal, but means pay people depending on their contribution. Some people are better at job than others. Some people contribute more than others. They deserve more money. 

- Take the issue of money off the table. For more creative, complex work, money alone is not the motivator. For your team to effectively work and stay motivated, pay them enough so they don't think about money, and can focus all their energies in pursuing the solutions to problems. 

For simple tasks, paying more will improve motivation, however this is not true for complex tasks. 

For enduring performance once above are satisfied there are 3 motivators:

1. Purpose: Do people know why they are doing something? Not merely how to do it. Are they making a difference, are they making a contribution? 

2. Mastery: are people getting better at something that matters? Are they making progress in meaningful work? 

3. Autonomy: Do people have a say? Do they have some control over what they are doing?

As a leader, it is your responsibility to ensure you create an environment where purpose, mastery and autonomy can blossom.

PURPOSE

- Purpose (Capital P). Making a difference to the world. Like solving climate crisis, feeding hungry. 

- purpose (Small p purpose). Are you simply making a contribution? Is my team making a difference?

Both purposes are important. Two interesting studies.

1. University of Michigan. Work study students working in a call centre to raise funds for the university. Turned out that when students spent 5 mintues reading a letter from people who were beneficieries of the funds, or even better meeting the beneficiaries, their performance increased significantly. This is the power of capital P purpose.

2. Study of Harvard business school cafeteria. Cooks cannot see customers and customers cannot see cooks. The study with the help of a rigged iPad enabled cooks to see customers and vice versa. The quality of the food improved. Cooks when saw their beneficieries increased their performance. 

Promoting purpose

Whose purpose is it anyway? using a blank card to pass around in the team asking them to write down the purpose. Turns out the results usually can be categorized into two groups - first where people are clueless, and second where there seems to be a common theme, a hum, or music that resonates the purpose of organization.

How to promote purpose?

1. By talking about why instead of how.

We always talking about how, but never why. Why is an enormous performance enhancer. Once people know why they are doing something, their performance automatically improves because humans are deeply motivated by a sense of purpose.

Have fewer conversations on how and more on why. Use why conversations than how conversations.

2. By building belonging in organizations. 

Let belonging emerge from the bottom like team members initiating activities like pot lucks, or someone bringing in cake/ bake items to intiaite the camaraderie and belongingness. Mandatory happy hours mandatory team lunches are forced fun that doesn't resnoate with the bottom, and therefore doesn't help nurture belongingness.

MASTERY

• Self feedback
• At end of everyday, take 60 secs to write down 3 things you got done that day.
• This will allow you to see the progress, and sustain the progress.
• Summon intrinsic motivation.
• The best feedback is frequent and informal.
• Traditional performance reviews do not do a good job of promoting mastery because they are infrequent, formal, and ineffective. Kabuki style, everything seems orchestrated.
• Weekly 1-1's with a twist (You are the manager, and you checkin your reportee)
• Week 1 checkin, how are you doing; what do you need.
• Week 2 checkin, how are you doing; what do you need.
• Week 3 checkin, how are you doing; what do you need.
• Twist: We talk about something diff. What do you love about your job, what you don;t
• Next month - check in, check in, check in, then talk about how to remove barries to doing your job effective. 
• 4th slot is customizable, continuous improvement.
• Give feedback on performance. Negative feedback. It is uncomfortable to deliver and receive.
• It is called WISE FEEDBACK.
• 19 words you can use to negative feedback as a source of mastery.
• I am giving you these comments because I have very high expectations, and I know that you can reach them.
• Do not shy away from negative feedback.

AUTONOMY

• Some jobs are inherently autonomous than others.
• Hospital Janitor research.
• Managers told the janitors 
• Craft the job in a way meaningful to you.
• If you want, go beyond your boundaries of what you are supposed to do if doing that makes you enjoying your job more.
• Saying hi, talking to patients, etc. etc.....help nurse, etc.
• Above made their job more meaningful, engaged more, were more productive, and stuck to their jobs longer.
• Every job deserves autonomy and self direction.
• As manager, make sure that happens.

How to promote autonomy

Ask:

1. Do my people have control in what they do?

2. Do they find it challenging?

If no to any one of above, you got work to do.

Autonomy Audit

Have your team rate it on a scale of 1 to 10.

• How much autonomy do you have over your tasks at work?
• How much autonomy do you have over your time at work? (when you show up and when you leave, how you allocate hours each day)
• How much autonomy do you have over your team at work? (How much say you have with people you work with collaborate with)
• How much autonomy do you over your technique at work  - how you actually perform the main responsibilities of your job.

Atlassian 

- Once a quarter, thu afternoon next 24 hours -- do whatever you want. But showcase it on Friday.

- These are called A Ship It Day. One day of intense undiluted autonomy

- Similar to hackathons, Google 20% time

PEOPLE DO GREAT WORK WHEN THEY HAVE PERIODS OF UNADULTERATED AUTONOMY

Autonomy is the pathway to innovation and creativity 

Reconfigure environments

Create islands of undiluted autonomy (1/2 hours of each week or 1 day a month kind of)

4. TRANSFROMATIONAL LEADERSHIP

Transformational leadership is a theory of leadership where a leader works with teams or followers beyond their immediate self-interests to identify needed change, creating a vision to guide the change through influence, inspiration, and executing the change in tandem with committed members of a group; This change in self-interests elevates the follower's levels of maturity and ideals, as well as their concerns for the achievement.[

This leader affects change through inspiring and motivating others to be more productive and involved.

Transformational leaders use four practices that help the people on their team perform at their best. Let's explore each of the four practices.

1. Lead with a purposeful mission for change
• Transformational leadership begins with a clear and purposeful mission for change. Leaders articulate a compelling vision that inspires and motivates their team members to commit to a common goal. They communicate this mission effectively, ensuring that every member of the organization understands the direction in which they are headed and the significance of their contributions. By providing a sense of purpose and direction, leaders foster a shared commitment to driving meaningful change, energizing their teams to overcome obstacles and pursue ambitious objectives.
2. Role model integrity
• Integrity lies at the heart of transformational leadership. Leaders lead by example, demonstrating honesty, transparency, and ethical behavior in all their actions. They uphold high moral standards and adhere to a strong sense of right and wrong, earning the trust and respect of their followers. By modeling integrity, leaders create a culture of accountability and credibility, where individuals feel confident in the integrity of their leaders and the organization as a whole. This commitment to ethical conduct fosters a positive work environment built on trust, honesty, and mutual respect.
3. Stay curious and inspire innovation
• Transformational leaders embrace a mindset of continuous learning and curiosity. They encourage their teams to question assumptions, explore new ideas, and challenge the status quo. By fostering a culture of curiosity and experimentation, leaders inspire innovation and creativity within the organization. They celebrate new approaches and encourage risk-taking, recognizing that innovation often requires stepping outside of comfort zones. Through their openness to new possibilities and willingness to embrace change, leaders create an environment where innovation thrives, driving ongoing improvement and adaptation.
4. Empower each to be their best.
• Empowerment is a cornerstone of transformational leadership. Leaders recognize the unique talents and strengths of each individual and empower them to reach their full potential. They provide opportunities for growth and development, offering support, guidance, and resources to help team members succeed. By delegating authority and decision-making responsibilities, leaders foster a sense of ownership and autonomy, empowering individuals to take initiative and make meaningful contributions. This culture of empowerment not only enhances individual performance but also cultivates a sense of ownership and commitment to the organization's goals, driving collective success.

Purposeful leadership

• Leader energizes a group of people to a shared goal. 
• Purpose is how you create a positive impact for others
• Vision is the outcome; the better future you want to create
• Discover other's strengths and energizers (what motivates and propels them forward). You may use assessment tools.

Managing resistence to change.

• Change is seen as a threat.
• Triggers fight or flight survival mechanism - 
• Fight: openly resist.
• Flight: disengage from participation.
• Freeze: inaction due to being confused or overwhelmed.
• Appease: a need to please that is motivated by fear, not real buy-in.

How to manage resistance to change?

• Create a stakeholder map.
• What is at stake for them? // how will change impact them?
• How do you see them reacting? Are they in fight or flight mode?
• Get your stakeholders out of fight or flight mode:
• Engage in powerful conversations that builds trust.
• Start with what is important to people.
• Common interests between you and them.
• Practice curiousity. Ask what is imp to you, what is the impact of this change on you?
• 
  

Check

• Have i built trust?
• Have i discoverd motivators and aspirations?
• Am i nurturing their confidence?
• Am i helping them become self aware?
• Am i bringing in curiosity?

Generative AI and LLMs for Dummies

Source: Generative AI and LLMs For Dummies®, Snowflake Special Edition

Traditional AI

Traditional AI is also referred to as Machine Learning (ML) focusses on analytical tasks like classification and prediction. 

Generative AI 

Generative AI goes a step further with its ability to create new, original content. 

Gen AI is a type of artificial intelligence that uses neural networks and deep learning algorithms to identify patterns within existing data as a basis for generating original content. By learning patterns from large volumes of data, gen AI algorithms synthesize knowledge to create original text, images, audio, video, and other forms of output.

Data Strategy (IBM)

 

Components of Data Strategy

(Some definitions are taken from IBM.com)

Data Pipeline: 

A data pipeline is a method in which raw data is ingested from various data sources, transformed and then ported to a data store, such as a data lake or data warehouse, for analysis.

Before data flows into a data repository, it usually undergoes some data processing. This is inclusive of data transformations, such as filtering, masking, and aggregations, which ensure appropriate data integration and standardization. This is particularly important when the destination for the dataset is a relational database. This type of data repository has a defined schema which requires alignment—that is, matching data columns and types—to update existing data with new data. 

Data can be sourced from:

- APIs

- SQL/Non-SQL databases

- Flat files

- Other formats

Before data flows into a data repository, it usually undergoes some data processing. This is inclusive of data transformations, such as filtering, masking, and aggregations, which ensure appropriate data integration and standardization. This is particularly important when the destination for the dataset is a relational database. This type of data repository has a defined schema which requires alignment—that is, matching data columns and types—to update existing data with new data. 

Type of data pipelines

a. Batch Processing

Batch processing loads “batches” of data into a repository during set time intervals, which are typically scheduled during off-peak business hours. This way, other workloads aren’t impacted as batch processing jobs tend to work with large volumes of data, which can tax the overall system. Batch processing is usually the optimal data pipeline when there isn’t an immediate need to analyze a specific dataset (for example, monthly accounting), and it is more associated with the ETL data integration process, which stands for “extract, transform, and load.”

Batch processing jobs form a workflow of sequenced commands, where the output of one command becomes the input of the next command. For example, one command might kick off data ingestion, the next command may trigger filtering of specific columns, and the subsequent command may handle aggregation. This series of commands will continue until the data quality is completely transformed and rewritten into a data repository.

We did it using Control-M scheduling where an out condition is passed on to the downstream application(s), which picks it up as an in condition to continue the ingestion, transformation, etc. 

b. Streaming data pipelines / event-driven architectures

Unlike batching processing, streaming data pipelines—also known as event-driven architectures—continuously process events generated by various sources, such as sensors or user interactions within an application. Events are processed and analyzed, and then either stored in databases or sent downstream for further analysis. 

Streaming data is leveraged when it is required for data to be continuously updated. For example, apps or point-of-sale systems need real-time data to update inventory and sales history of their products; that way, sellers can inform consumers if a product is in stock or not. A single action, such as a product sale, is considered an “event,” and related events, such as adding an item to checkout, are typically grouped together as a “topic” or “stream.” These events are then transported via messaging systems or message brokers, such as the open-source offering, Apache Kafka. 

Since data events are processed shortly after occurring, streaming processing systems have lower latency than batch systems, but aren’t considered as reliable as batch processing systems as messages can be unintentionally dropped or spend a long time in queue. Message brokers help to address this concern through acknowledgements, where a consumer confirms processing of the message to the broker to remove it from the queue.

Data Pipeline Architecture

Remember DDEP zones - 

Raw zone or Landing zone: (no user access, true source data stored in source format, history is retained), 

Democratized zone: Source with field level encryption, natural keys extracted to enable integration, history is retained. Seamless access to enterprise data without the overhead of stringent and bureaucratic access controls. Sensitive data is encrypted.

Publish zone: Operational data sets. Downstream applications connect to source data; SLA driven. Data structures are designed to meet consumption patterns. Publish data is consumed by downstream apps, APIs, EDW, and Bi Analytics tools.

Discovery zone: is another zone beneath the democratized zone from where data science tools and BI/analytics tools connect to derive their individual needs.

FDP/BDP/XDP: Data Product: are additional layers to the right of Publish from where applications source data for individual needs. 

> Ingestion: data is ingested into raw and from there to publish/bdps where it is stored. 

> Transformation: transformation happens prior to moving data to publish and bdps. 

> Storing: data is stored

 

When sending downstream, the data is sent outbound via outbound job on to Sterling or another system via passing out condition.

ETL Vs. Data Pipelines

An ETL Pipeline ends with loading the data into a database or data warehouse. A Data Pipeline doesn't always end with the loading. In a Data Pipeline, the loading can instead activate new processes and flows by triggering webhooks in other systems.

Data Lineage

Data lineage is the process of tracking the flow of data over time, providing a clear understanding of where the data originated, how it has changed, and its ultimate destination within the data pipeline.

Data lineage tools provide a record of data throughout its lifecycle, including source information and any data transformations that have been applied during any ETL or ELT processes.

This type of documentation enables users to observe and trace different touchpoints along the data journey, allowing organizations to validate for accuracy and consistency. This is a critical capability to ensure data quality within an organization. It is commonly used to gain context about historical processes as well as trace errors back to the root cause.

Reliable data is essential  to drive better decision-making and process improvement across all facets of business--from sales to human resources. However, this information is valuable only if stakeholders remain confident in its accuracy as insights are only as good as the quality of the data. Data lineage gives visibility into changes that may occur as a result of data migrations, system updates, errors and more, ensuring data integrity throughout its lifecycle.

Data lineage documents the relationship between enterprise data in various business and IT applications.

Datawarehouse vs. Data Lake vs. Data Mart

A data warehouse is a system that aggregates data from multiple sources into a single, central, consistent data store to support data mining, artificial intelligence (AI), and machine learning—which, ultimately, can enhance sophisticated analytics and business intelligence. Through this strategic collection process, data warehouse solutions consolidate data from the different sources to make it available in one unified form. 

A data mart is a focused version of a data warehouse that contains a smaller subset of data important to and needed by a single team or a select group of users within an organization. A data mart is built from an existing data warehouse (or other data sources) through a complex procedure that involves multiple technologies and tools to design and construct a physical database, populate it with data, and set up intricate access and management protocols.

While it is a challenging process, it enables a business line to discover more-focused insights quicker than working with a broader data warehouse data set. For example, marketing teams may benefit from creating a data mart from an existing warehouse, as its activities are usually performed independently from the rest of the business. Therefore, the team doesn’t need access to all enterprise data.

A data lake, too, is a repository for data. A data lake provides massive storage of unstructured or raw data fed via multiple sources, but the information has not yet been processed or prepared for analysis. As a result of being able to store data in a raw format, data lakes are more accessible and cost-effective than data warehouses. There is no need to clean and process data before ingesting.

For example, governments can use technology to track data on traffic behavior, power usage, and waterways, and store it in a data lake while they figure out how to use the data to create “smarter cities” with more efficient services.

Data Identification:

Critical Data Elements

Every organization handles a vast volume of data, but not all data are equally crucial to their objectives. They prioritize data governance based on business goals, regulatory requirements, and risk tolerance. By focusing efforts on critical data, particularly Critical Data Elements (CDEs), they effectively manage data risks as part of our Risk Management Strategy.

Data Element is a unit of data. Critical Data Elements are those that if missed or of low quality will impact a business' ability to carry out business.  

Data Storage

Data Provisioning

Data Integration

Data Governance

Data and Data Science

 

SAP Mass Transport (Sony UK 2012 to 2016)

 In the context of SAP (Systems, Applications, and Products in Data Processing), a "mass transport" refers to the process of moving changes or developments from one SAP system to another. This is typically done using the Transport Management System (TMS), a tool provided by SAP for managing the transportation of changes across various SAP landscapes, such as development, quality assurance, and production environments.

Here's how the process generally works:

Development: Changes are made in the development environment, such as creating or modifying programs, reports, configurations, or customizations.

Transport Request Creation: Once the changes are completed and tested in the development system, they are bundled together into a transport request. This request contains all the necessary information and objects required to implement the changes in other systems.

Release of Transport Request: The transport request is then released by an authorized person. Releasing the transport request signifies that the changes are ready to be moved to other systems.

Importing into Other Systems: The released transport request is imported into other systems, such as quality assurance or production environments, using the Transport Management System. During this process, the system checks dependencies and ensures that the changes are applied correctly.

Testing: After the changes are imported into the target systems, thorough testing is performed to ensure that the functionality works as expected and that there are no adverse effects on other processes.

Approval and Confirmation: Once testing is successful, the changes are approved, and confirmation is sent back to the development team.

Documentation: It's essential to maintain proper documentation throughout the process, including documenting the changes made, the transport requests created, and any issues encountered and their resolutions.

By using mass transport, SAP enables organizations to maintain consistency and integrity across their SAP landscapes while facilitating efficient development and deployment processes.