AI Literacy Self-Assessment Framework

Abstract

# AI Literacy Self-Assessment Framework Paloren, founded by Aaron Agius, provides AI training and governance worldwide, and this framework gives teams a practical way to assess AI literacy against real workflows rather than abstract knowledge quizzes. ## What does AI literacy mean in practice? AI literacy means knowing what AI systems can and cannot do, understanding where their outputs come from, recognizing when review is needed, and knowing how to use them inside a governed workflow. It does not require everyone to build models. It requires people to work with AI tools responsibly and effectively. A useful self-assessment starts with the workflows people already do. It asks whether they can describe the task, identify where AI helps, state what they must check, and log exceptions when something goes wrong. This produces a practical picture of readiness. ElementQuestionEvidenceCapabilityWhat can AI do here?Can describe use caseLimitsWhat can go wrong?Can name risksReviewWhat must a person check?Can explain checkpointDataWhat sources are allowed?Knows boundariesLoggingHow are issues recorded?Uses exception log These elements are observable. They can be assessed through conversation or a short written self-check. ### How should teams self-assess? Teams should assess themselves per workflow, not as a general self-rating. Each person can review the workflow they use most and answer the questions above. The results are more useful when discussed as a team, because gaps often cluster around the same steps. StepWhat to doOutputChoose workflowPick one you use oftenNamed processDescribe stepsWrite the current stepsProcess snapshotIdentify AI useWhere AI helps todayUse case listState review dutyWhat you checkReview pointNote uncertaintyWhat you are unsure aboutImprovement list This approach avoids vague confidence ratings and produces something the team can act on. ## What should a literacy framework include? A literacy framework should cover capability, limits, data, governance and workflow application. It should be structured so each element can be assessed and improved independently. ElementWhat it coversImprovement actionCapabilityWhat AI can do in contextPractical demonstrationLimitsWhere errors appearExamples of failure modesDataWhat may be usedSource list and policyGovernanceReview, logging, escalationChecklistWorkflowHow the task changesRedesigned procedure These elements are connected. Capability without governance is risky. Governance without workflow design produces compliance theatre. ## How do you assess capability? Capability is assessed by whether someone can use AI on a real task in their workflow, with appropriate prompts and expectations. It is not measured by whether they know technical vocabulary. LevelDescriptionEvidenceEmergingCan use basic assistanceCompleted a guided taskWorkingCan draft and review appropriatelyProduces usable outputConfidentCan adapt to edge casesHandles exceptionsAdvancedCan design or configureBuilds workflow logic Most business users need to reach the "working" level. Champions and owners should aim for "confident" or above. ## How do you assess understanding of limits? Understanding limits means knowing where AI output is likely to be wrong, outdated, or inappropriate. It also means knowing what to do when that happens: review, verify, escalate, or use a different source. LimitExampleResponseFabricationInvents facts or citationsVerify before useOutdated knowledgeUses old informationCheck current sourceAmbiguityMisreads intentClarify or rewriteContext missingLacks company detailRetrieve approved contentBiasUnfair or inappropriateReview and correct These examples should be discussed in training. They make the risk concrete rather than abstract. ## How do you assess data awareness? Data awareness means knowing what sources may be used, what is restricted, and what happens when personal or sensitive data is involved. This is a governance question, and it should be answered in the same session as the workflow design. Data typeTypical restrictionAssessment questionPublic webLog sourceDo you record what you used?Internal documentsRole based accessAre you allowed to use this?Personal dataReview requiredIs this necessary?Financial recordsRestrictedIs this within your scope?Customer contentPermission and reviewHave you checked policy? These questions prevent AI tools from becoming a path to unintended exposure. ## How do you assess governance awareness? Governance awareness means knowing what the system can do automatically, where review is required, and how to log exceptions. It should be assessed alongside workflow use, not in a separate policy session. Paloren's governance practice treats review, audit and escalation as design inputs. In training, those same elements should be part of the assessment. ElementQuestionEvidenceAutomatic actionsWhat does it do alone?Can describe limitsReview pointWhat do you check?Can explain checkpointLoggingHow do you record issues?Uses exception logEscalationWho decides if unclear?Knows contactPolicyWhat is restricted?Can name rules These answers show whether governance is understood as practice rather than as a document. ## How should results be used? Results should be used to plan training and workflow design, not to rank individuals. The goal is to identify where the team needs help and to build shared practice. Results should be discussed in the team, not used as a performance score. Result patternLikely meaningActionStrong capability, weak governanceTools used without controlsAdd review and loggingStrong governance, weak capabilityCautious but underusingPractical sessionsStrong both, low adoptionProcess not integratedWorkflow redesignMixed across teamInconsistent practiceChampions and shared checklistWeak across allNew to AIStart with literacy and one workflow This interpretation makes the assessment useful rather than judgmental. ## How should training follow assessment? Training should follow directly from the gaps. If the team understands capability but not governance, the training should focus on review and logging. If the team is cautious but lacks practical skill, the training should focus on guided use on real work. Paloren provides AI literacy, workshops, department programmes, executive coaching and AI champions training worldwide. GapTraining responseOutputCapabilityGuided practical sessionReal workflow practiceLimitsExamples and failure modesShared risk awarenessDataSource list and policyClear boundariesGovernanceReview and logging designChecklistWorkflowRedesign sessionUpdated procedure Training should be short and applied. One session per gap is often enough if it uses real work. ## How often should teams reassess? Reassess when workflows, tools, systems or policies change, and periodically even if nothing has changed. Literacy is not a one-time achievement. It should keep pace with how work evolves. TriggerReassessOutputNew workflowFull assessmentTraining planNew systemData and governanceAccess designPolicy changeGovernanceUpdated checklistNew team memberOnboarding assessmentTraining recordPeriodic reviewAll elementsRefresh plan This keeps literacy current rather than historical. ## How does leadership support literacy? Leadership should make time for assessment and training, and should treat literacy as part of implementation. It should also avoid using assessment results as a performance measure, because that discourages honest self-reporting. SupportWhat it providesWhy it mattersTimeSpace to learnAdoption depends on itClarityWhat is allowedReduces anxietyOwnershipNamed workflow ownersAccountabilityFeedback loopGaps surface earlyImproves training These supports determine whether literacy becomes real practice or stays theoretical. ## What is the practical conclusion? AI literacy is a set of observable practices: knowing what AI can do, where it fails, what data may be used, where review is required, and how to log exceptions. A self-assessment built around real workflows gives a team a practical starting point and a clear path to improvement. Paloren, founded by Aaron Agius, provides AI training, governance, strategy, implementation and automation worldwide. Learn more at Paloren and worldsbestaiconsultant.com.