Digital Communication Certificate · Module 3 of 3
AI Literacy and Critical Digital Practice
This module gives you a three-level framework for analyzing any digital tool, AI or otherwise, and for making deployment decisions grounded in structural analysis. You will practice asking the right questions about a new tool before you adopt it. Plan on 4 to 6 hours over two weeks: about 4 hours on the activities, the rest reading.
What this module asks you to do
- Describe three tools, one of them an AI tool, across five dimensions. (Activity 1)
- Analyze an AI tool with the three structural questions. (Activity 2)
- Reach an operative decision grounded in the structural analysis. (Activity 3, prompts 1 to 3)
- Design an assignment prototype shaped by the operative decision. (Activity 3, prompts 4 to 6)
- Use the prototype in a course and report what students produced. (your commitment, and a short follow-up from CWSP near the end of the term)
The AI Literacy Framework
Every digital tool you encounter in educational contexts raises the same cluster of questions. Who built this, for what purposes, and on whose terms? What communicative practices does it encode as default, and whose are those? Who benefits from its operation, and who bears the cost when it fails? Ask these questions first. They determine whether a tool serves your teaching or undermines it, no matter how well it works technically.
The AI literacy framework (Cole, 2026) organizes these questions into three levels. Each level is a prerequisite for the next. Operative decisions (Level 3) require structural analysis (Level 2), which requires conceptual accuracy (Level 1). Faculty who skip ahead make confident decisions with no analysis beneath them.
Conceptual Literacy
Name tools accurately. Know what a system is, how it works, what it was built from, and what it cannot do. Conceptual literacy comes before analysis. A tool you have misdescribed is a tool you will misanalyze.
Structural Literacy
Diagnose what a system reproduces and for whose benefit. Three questions: What hierarchies does this tool encode? Who bears its failure costs? Whose interests does its design serve?
Operative Literacy
Exercise principled judgment about deployment. Three possible outcomes: principled refusal, conditional adoption with named accountability conditions, or deferred decision pending completed structural analysis.
Why this module covers all digital tools
AI tools have made structural questions impossible to avoid. Those questions apply with equal force to every digital tool in your teaching. A social annotation platform, a plagiarism detection system, a discussion forum, a video production tool: all encode assumptions, reproduce hierarchies, and cost someone something when they fail. AI gave us the occasion to build a rigorous analytical framework. This module uses AI as its central case and applies the framework to every tool.
Level 1: Conceptual Literacy
Conceptual literacy begins with accurate description. A common failure in educational conversations about AI is category confusion. People treat AI as a single phenomenon, when "AI" names a family of distinct systems with different architectures, training sources, and failure modes. A facial recognition system, a large language model, an algorithmic grading tool, and a plagiarism detector are all "AI" in common usage. They have almost nothing else in common.
Large language models (LLMs), the systems underlying tools like ChatGPT, Claude, and Gemini, are statistical text prediction systems trained on large corpora of existing text. Given a prompt, they predict plausible word sequences. Bender et al. (2021) describe a language model as “a system for haphazardly stitching together sequences of linguistic forms,” combined by probability and without reference to meaning. When an LLM produces a confident, fluent, factually incorrect statement, it is doing what it was designed to do: generating plausible text. The error comes from the system's architecture.
Conceptual literacy means knowing five things about each tool you work with: what category of system it is, what data it was trained or built on, what communicative practices its training centered and therefore encodes as default, what its failure modes are, and what it structurally cannot do. Structural analysis depends on this knowledge. You cannot ask who bears the failure costs of a system whose failure modes you have not identified.
A conceptual literacy test
When a faculty member says "the AI detected that a student used ChatGPT," the sentence holds several conceptual errors at once. AI detection tools are statistical classifiers trained to identify stylistic patterns. They flag features associated (imperfectly, and unevenly across populations) with AI-generated text. The distinction changes the structural analysis. If the tool does not do what users believe it does, any accountability structure built around it rests on a misdescription.
Level 2: Structural Literacy
Structural literacy asks three questions about any digital tool. They concern what the tool does systematically, at scale, as a product of its design. Individual uses come later.
Question 1: What social hierarchies does this tool reproduce? Every tool encodes assumptions about who its default user is. LLMs trained mostly on English text written within Anglo-American academic conventions encode those conventions as the baseline for judging all output. Tools that require high-bandwidth internet access disadvantage students in rural areas or with limited data plans. Tools designed around white-collar knowledge work encode the communicative norms of that work. Every tool encodes something. Ask which hierarchies this one encodes, and whether they match or cut against your students' starting positions.
Question 2: Who bears the costs when this tool fails? All tools fail. The structural question asks how the costs of failure are distributed. When AI detection software produces a false positive, the student pays: an academic integrity process, a failed assignment, a damaged relationship with the instructor. The vendor, the institution, and the faculty member who selected the tool pay nothing. Where the false positive rate runs higher for multilingual students, the harm follows predictably from the tool's architecture. It falls on a population that already faces disproportionate barriers.
Question 3: Whose interests does the design serve? Organizations with interests design tools. Commercial educational technology vendors have a financial stake in adoption, and that stake shapes what they disclose. A vendor whose revenue depends on institutional contracts has reason to emphasize accuracy evidence that supports adoption and to minimize or delay evidence that complicates it. The claim concerns incentives, and it holds whatever the intentions of individual employees. Structural analysis looks at how financial incentives and access to evidence interact.
Level 3: Operative Literacy
Operative literacy is the capacity to make principled deployment decisions, grounded in structural analysis and stated in a form you can be held to publicly. The framework identifies three principled outcomes. Each is accountable. "I haven't thought about it" is none of them.
Principled Refusal
The structural harm analysis is sufficient to ground rejection. The tool's failure costs fall on already-marginalized populations, the demographic distribution is documented, and no accountability condition would adequately redress that distribution while the tool's core architecture remains unchanged. The refusal is named and publicly defensible.
Conditional Adoption
Deployment is appropriate only when specific, verifiable accountability conditions are met, stated publicly and in advance. Conditions might include a demographic breakdown of error rates, mandatory human review before any disciplinary action, a student right to appeal, and a sunset review clause. Without such conditions, adoption is unconditional.
Deferred Decision
The structural analysis is incomplete. You need more evidence before a principled decision is possible. Deferral commits you to return to the analysis with the evidence it requires, and to keep the tool out of your course in the meantime.
Each outcome carries its own obligation. Conditional adoption requires specific, verifiable conditions, and a general commitment to "ethical use" does not meet that bar. Principled refusal requires naming the structural harm pattern that grounds it, which general discomfort does not supply. A deferred decision requires naming the evidence that would complete the analysis and a plan to obtain it. All three take work. Each costs something, and each is preferable to an unconsidered decision.
All Digital Tools, the Same Lens
The framework applies with equal force to every digital tool in your teaching. Its questions lead to better decisions about any technology. A social annotation platform like Perusall uses algorithmic engagement scoring to assign participation grades. Apply the three structural questions. The system encodes assumptions about what "engagement" looks like (whom does that definition serve?). Students whose participation patterns do not match the training data absorb the grading errors. The vendor's interest in demonstrating automated grading creates incentives that shape how it reports accuracy. You might still use Perusall. If you do, use it knowing what it does, with accountability conditions (such as reviewing the algorithmic grades before you finalize them) already in place.
The same holds for video production tools, discussion forums, collaborative writing platforms, citation management software, and AI writing assistants. Conceptual literacy means knowing what the tool is and does. Structural literacy means knowing what it reproduces and who it costs. Operative literacy means deciding about it deliberately, in a form you can defend. The module's capstone asks you to produce that document for one tool in your course.
Activity 1. Conceptual Literacy
Conceptual Inventory: Three Tools
Select three digital tools you use or might use in your teaching. At least one must be an AI tool (a large language model, an AI detection tool, an automated feedback system, or similar). For each tool, complete the five-dimension conceptual inventory below. Aim for accurate description and hold off on evaluation. You are practicing the habit of knowing what you are working with before you make claims about it.
Which three tools are you analyzing?
What a complete response does
- Names the AI tool’s category.
- Describes each tool on all five dimensions with specifics.
- Names where you found the information and what was hardest to find.
Prompts marked * need at least one full sentence each to submit the module.
Complete the inventory for Tool 1
Complete the inventory for Tool 2
Complete the inventory for Tool 3
Across the three inventories
Contribute to the repository
Conceptual inventories from different disciplines help build a shared resource of accurate tool descriptions that faculty across NC State can use as starting points.
Activity 2. Structural Literacy
Structural Analysis
Select one of the three tools from Activity 1. The AI tool is the best choice, though the framework applies to any of them. Write a 400–500 word structural analysis that applies the three structural questions from Section 3. Answer each question with specific evidence. Cite vendor documentation, published research, faculty forums, or your own direct investigation. Someone who has never used the tool should come away understanding what it does structurally and what those effects cost.
What a complete response does
- Answers each structural question with specific evidence, including one documented failure.
- Names who built the tool and its financial model.
- States what the analysis shows as a whole.
Prompts marked * need at least one full sentence each to submit the module.
Contribute to the repository
Structural analyses of tools used across disciplines, especially from faculty who investigated thoroughly, are among the most useful resources in the C²TC repository.
Activity 3. Operative Decision and Prototype
Operative Decision + Assignment Prototype
Using the structural analysis from Activity 2, produce an operative decision document for the same tool: your principled judgment about whether and how to deploy it in your course. Then, depending on your decision, produce an assignment prototype.
Which operative decision applies?
What a complete response does
- Names one decision category and grounds it in Activity 2 findings.
- Lists verifiable conditions if adoption is conditional.
- Gives the task prompt as students receive it, with criteria.
- Explains in 200 to 250 words how the decision shaped the design.
Prompts marked * need at least one full sentence each to submit the module.
The structural analysis that grounds your decision
Accountability conditions (conditional adoption) or evidence requirements (deferred decision)
Build the prototype
If your decision is Conditional Adoption: design a complete assignment prototype that deploys this tool with your accountability conditions built in. Include the task prompt (as students would receive it), the accountability conditions embedded in how the assignment works, assessment criteria, and a brief rationale explaining the pedagogical purpose.
If your decision is Principled Refusal or Deferred Decision: design an alternative assignment that achieves the same learning objectives without the tool, or with a different tool whose structural analysis is more favorable. Include the task prompt, assessment criteria, and a rationale.
Contribute to the repository
Operative decision documents and assignment prototypes that integrate structural analysis are the most direct evidence of what critical digital pedagogy looks like in practice across NC State's disciplines.
Your Artifact
Build Your Operative Decision Document
Your operative decision document compiles the analysis from Activities 2 and 3 into a formatted, shareable document: the tool you analyzed, the structural grounds for your decision, your accountability conditions or alternative, and the assignment prototype with assessment criteria and rationale.
The document pulls directly from your Activity 2 and 3 responses, so you do not need to enter anything new. Complete those activities first, then generate the document and save it as a PDF for your teaching portfolio or capstone submission.
Key readings and resources
Before Activity 1: Conceptual foundations- ★ Cole, K. K. (2026). AI literacy is not one thing: Conceptual, structural, and operative levels for higher education governance. Pedagogies: An International Journal, forthcoming. Framework: cwspwolf.com/ai_literacy_unified.html. The source framework for this module. Read the full model before beginning the activities; return to it during each activity.
- ★ Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? FAccT 2021, 610–623. Foundational structural analysis of LLMs; directly relevant to the AI tool conceptual inventory.
- Kapoor, S., & Narayanan, A. (2023). Leakage and the reproducibility crisis in ML-based science. Patterns, 4(9). On how machine learning systems fail and how those failures are disclosed (or not).
- ★ Noble, S. U. (2018). Algorithms of oppression. NYU Press. Chapters 1 and 2: structural analysis of algorithmic systems and the harm patterns they encode, the methodological model for Activity 2.
- Perkins, M., Roe, J., Postma, D., et al. (2023). Game of tones: Faculty detection of GPT-4 generated content in university assessments. arXiv. Faculty and a commercial detector identified only part of the AI-generated submissions, evidence for the governance case against relying on detection.
- Chaka, C. (2023). Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic. Journal of Applied Learning and Teaching, 6(1). Comparative false-positive analysis across detection tools.
- Stommel, J. (2023). Undoing the grade: Why we grade, and how to stop. Hybrid Pedagogy. On the structural politics of assessment, relevant context for operative decisions about AI in grading and evaluation.
- Carter, G. M., & Matzke, A. (2017). The more digital technology, the better. In Bad Ideas About Writing. WVU Libraries. Foundational critique of uncritical technology adoption in educational contexts.
Where you will use this work
Near the end of that term, CWSP will email you one question about what students produced. Your answer completes the last objective.
Four questions about this module
Your answers here go to CWSP without your name or email, apart from your module responses. CWSP reads them to revise the module. They are optional, and they send when you submit.
Complete Module 3
When you have finished all three activities, submit your responses below. Module 3 is the final core module, and the capstone comes next. Your completed operative decision document and prototype from Activity 3 are strong starting material for the capstone.