Digital Communication Certificate · Module 3 of 3
AI Literacy and Critical Digital Practice
A three-level framework for analyzing any digital tool — AI or otherwise — and producing principled deployment decisions grounded in structural analysis. The goal: faculty who ask the right questions about every new tool before pressing adopt. Estimated time: 4–5 hours over two weeks.
The AI Literacy Framework
Every digital tool you encounter in educational contexts raises the same cluster of questions. Who built this, for what purposes, and on whose terms? What communicative practices does it encode as default, and whose are those? Who benefits from its operation, and who bears the cost when it fails? These are not afterthought questions. They determine whether a tool serves pedagogical purposes or undermines them, regardless of how well it works technically.
The AI literacy framework developed by Professor Kirsti K. Cole (2026) organizes these questions into three levels, each building on the previous. The levels are not optional stages — they are a prerequisite structure. Operative decisions (Level 3) require structural analysis (Level 2), which requires conceptual accuracy (Level 1). Skipping ahead produces decisions that are technically confident but analytically groundless.
Conceptual Literacy
Name tools accurately. Know what a system is, how it works, what it was built from, and what it cannot do. Conceptual literacy is the precondition for analysis — you cannot analyze what you have misdescribed.
Structural Literacy
Diagnose what a system reproduces and for whose benefit. Three questions: What hierarchies does this tool encode? Who bears its failure costs? Whose interests does its design serve?
Operative Literacy
Exercise principled judgment about deployment. Three possible outcomes: principled refusal, conditional adoption with named accountability conditions, or deferred decision pending completed structural analysis.
Why this module focuses on all digital tools, not only AI
The emergence of AI tools has made structural questions impossible to avoid — but those questions apply with equal force to every digital tool in your teaching practice. A social annotation platform, a plagiarism detection system, a discussion forum, a video production tool: all encode assumptions, reproduce hierarchies, and cost someone something when they fail. AI has given us the occasion to build a rigorous analytical framework. This module uses AI as the central case while insisting the framework is general.
Level 1: Conceptual Literacy
Conceptual literacy begins with accurate description. The most common failure in educational conversations about AI is category confusion — treating AI as a singular phenomenon when "AI" names a family of distinct systems with radically different architectures, training sources, and failure modes. A facial recognition system, a large language model, an algorithmic grading tool, and a plagiarism detector are all "AI" in common usage. They have almost nothing else in common.
Large language models (LLMs) — the systems underlying tools like ChatGPT, Claude, and Gemini — are statistical text prediction systems trained on large corpora of existing text. They predict plausible word sequences given a prompt; they do not retrieve stored knowledge, reason from principles, or hold beliefs. When an LLM produces a confident, fluent, factually incorrect statement, it is doing exactly what it was designed to do: generating plausible-looking text. That is not a malfunction. It is the system's architecture.
Conceptual literacy means knowing, for each tool you work with: what category of system it is; what data it was trained or built on; what communicative practices its training centered and therefore encodes as default; what its failure modes are; and what it structurally cannot do. This knowledge is the prerequisite for structural analysis. You cannot ask who bears the failure costs of a system whose failure modes you haven't identified.
A conceptual literacy test
When a faculty member says "the AI detected that a student used ChatGPT," multiple conceptual failures may be present simultaneously: AI detection tools are not AI systems detecting AI; they are statistical classifiers trained to identify stylistic patterns. They do not detect AI — they flag features associated (imperfectly, in specific populations) with AI-generated text. The distinction matters because it changes the structural analysis: if the tool does not do what users believe it does, the accountability structure built around it is built on a misdescription.
Level 2: Structural Literacy
Structural literacy asks three questions about any digital tool. These are not questions about individual uses of the tool — they are questions about what the tool does systematically, at scale, as a product of its design.
Question 1: What social hierarchies does this tool reproduce? Every tool encodes assumptions about who its default user is. LLMs trained predominantly on English text produced within Anglo-American academic conventions encode those conventions as the baseline against which all output is judged. Tools that require high-bandwidth internet access disadvantage students in rural areas or with limited data plans. Tools designed around white-collar knowledge work encode the communicative norms of that labor category. The question is not whether encoding happens — it always does — but which hierarchies are encoded and whether they correspond to or cut against your students' starting positions.
Question 2: Who bears the costs when this tool fails? All tools fail. The structural question is not whether failure occurs but how its costs are distributed. When AI detection software produces a false positive, the cost is borne by the student — an academic integrity process, a failed assignment, a damaged relationship with the faculty member — not by the vendor, the institution, or the faculty member who selected the tool. When that false positive rate is higher for multilingual students, the structural harm is not incidental. It is a predictable outcome of the tool's architecture, concentrated in a population that already faces disproportionate barriers.
Question 3: Whose interests does the design serve? Tools are designed by organizations with interests. Commercial educational technology vendors have financial interests in adoption that create incentive structures around disclosure. A vendor whose revenue depends on institutional contracts has structural incentives to emphasize accuracy evidence that supports adoption and minimize or delay disclosure of accuracy evidence that complicates it. This is not a claim about individual bad actors. It is a structural analysis of how financial incentives and epistemic access interact.
Level 3: Operative Literacy
Operative literacy is the capacity to make principled deployment decisions — decisions grounded in structural analysis and expressed in a form that is publicly accountable. The framework identifies three principled outcomes. All three are accountable; none of them is "I haven't thought about it."
Principled Refusal
The structural harm analysis is sufficient to ground rejection. The tool's failure costs fall on already-marginalized populations, the demographic distribution is documented, and no accountability condition could adequately redress that distribution while the tool's core architecture remains unchanged. The refusal is named and publicly defensible.
Conditional Adoption
Deployment is appropriate only when specific, verifiable accountability conditions are met — stated in advance, publicly. Conditions might include: demographic breakdown of error rates; mandatory human review before any disciplinary action; student right to appeal; sunset review clause. Adoption without these conditions is not conditional adoption. It is adoption.
Deferred Decision
The structural analysis is incomplete. Additional evidence is needed before a principled decision is possible. Deferral is not delay for its own sake — it is a commitment to return to the analysis with the evidence it actually requires, and not to deploy in the meantime.
Note the asymmetry: conditional adoption requires naming specific, verifiable conditions, not general commitments to "ethical use." Principled refusal requires naming the structural harm pattern that grounds it, not expressing general discomfort. Deferred decision requires identifying what evidence would complete the analysis, not indefinitely postponing a decision. All three require work. None of them is costless, and all of them are preferable to an unconsidered one.
All Digital Tools, the Same Lens
The framework applies with equal force to every digital tool in your teaching practice — not as a requirement to be suspicious of all technology, but as a set of questions that produce better decisions about any technology. A social annotation platform like Perusall uses algorithmic engagement scoring to assign participation grades. Apply the three structural questions: the system encodes assumptions about what "engagement" looks like (who does that definition serve?); algorithmic grading errors are borne by students whose participation patterns don't match the training data; and the vendor's interest in demonstrating automated grading capacity creates incentives around how accuracy is reported. None of this means you shouldn't use Perusall. It means you use it with specific knowledge of what it does, and with accountability conditions — like reviewing the algorithmic grades before finalizing them — already in place.
The same applies to video production tools, discussion forums, collaborative writing platforms, citation management software, and AI writing assistants. Conceptual literacy means knowing what the tool actually is and does. Structural literacy means knowing what it reproduces and who it costs. Operative literacy means deciding about it deliberately, in a form you can defend. The module's capstone asks you to produce that document for one tool in your course.
Activity 1 — Conceptual Literacy
Conceptual Inventory: Three Tools
Select three digital tools you use or could use in your teaching. At least one must be an AI tool (a large language model, an AI detection tool, an automated feedback system, or similar). For each tool, complete the five-dimension conceptual inventory below. The goal is not evaluation — it is accurate description. You are practicing the discipline of knowing what you are working with before you make claims about it.
Which three tools are you analyzing?
Complete the inventory for Tool 1
Complete the inventory for Tool 2
Complete the inventory for Tool 3
Across the three inventories
Contribute to the repository
Conceptual inventories from different disciplines help build a shared resource of accurate tool descriptions that faculty across NC State can use as starting points.
Activity 2 — Structural Literacy
Structural Analysis
Select one of the three tools from Activity 1 — ideally the AI tool, but the framework applies to any of them. Write a 400–500 word structural analysis applying the three structural questions from Section 3. Each question requires specific evidence, not general impressions. Cite vendor documentation, published research, faculty forums, or your own direct investigation. The analysis should be specific enough that someone who hasn't used the tool could understand what it does structurally and what those structural effects cost.
Contribute to the repository
Structural analyses of tools used across disciplines — especially from faculty who investigated deeply — are among the most valuable resources in the C²TC repository.
Activity 3 — Operative Literacy (Capstone)
Operative Decision + Assignment Prototype
Using the structural analysis from Activity 2, produce an operative decision document for the same tool — your principled judgment about whether and how to deploy it in your course. Then, depending on your decision, produce an assignment prototype.
Which operative decision applies?
The structural analysis that grounds your decision
Accountability conditions (conditional adoption) or evidence requirements (deferred decision)
Build the prototype
If your decision is Conditional Adoption: design a complete assignment prototype that deploys this tool with your accountability conditions built in. Include the task prompt (as students would receive it), the accountability conditions embedded in how the assignment works, assessment criteria, and a brief rationale explaining the pedagogical purpose.
If your decision is Principled Refusal or Deferred Decision: design an alternative assignment that achieves the same learning objectives without the tool — or with a different tool whose structural analysis is more favorable. Include the task prompt, assessment criteria, and a rationale.
Contribute to the repository
Operative decision documents and assignment prototypes that integrate structural analysis are the most direct evidence of what critical digital pedagogy looks like in practice across NC State's disciplines.
Your Artifact
Build Your Operative Decision Document
Your operative decision document compiles the analysis from Activities 2 and 3 into a formatted, shareable document: the tool you analyzed, the structural grounds for your decision, your accountability conditions or alternative, and the assignment prototype with assessment criteria and rationale.
The document pulls directly from your Activity 2 and 3 responses — no additional input needed. Complete those activities first, then generate the document and save it as a PDF for your teaching portfolio or capstone submission.
Key readings and resources
Before Activity 1 — Conceptual foundations- ★ Cole, K. K. (2026). AI Literacy Model. cwspwolf.com/ai_literacy_unified.html. The source framework for this module. Read the full model before beginning the activities; return to it during each activity.
- ★ Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? FAccT 2021, 610–623. Foundational structural analysis of LLMs; directly relevant to the AI tool conceptual inventory.
- Kapoor, S., & Narayanan, A. (2023). Leakage and the reproducibility crisis in ML-based science. Patterns, 4(9). On how machine learning systems fail and how those failures are disclosed (or not).
- ★ Noble, S. U. (2018). Algorithms of oppression. NYU Press. Chapters 1 and 2: structural analysis of algorithmic systems and the harm patterns they encode — the methodological model for Activity 2.
- Perkins, M., et al. (2023). Game of trones: Large language models do not detect non-English AI-generated text. arXiv. Empirical documentation of AI detection failure patterns by language — the failure-cost distribution evidence for the governance case.
- Chaka, C. (2023). Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic. Journal of Applied Learning and Teaching, 6(1). Comparative false-positive analysis across detection tools.
- Stommel, J. (2023). Undoing the grade: Why we grade, and how to stop. Hybrid Pedagogy. On the structural politics of assessment — relevant context for operative decisions about AI in grading and evaluation.
- Carter, G. M., & Matzke, A. (2017). The more digital technology, the better. In Bad Ideas About Writing. WVU Libraries. Foundational critique of uncritical technology adoption in educational contexts.
- CWSP Resource: Writing WITH AI, Not FOR AI — Cole, K. K. CWSP Framework Document. Discipline-specific guidance for integrating AI in writing courses; model for the conditional adoption prototype.
Complete Module 3
When you have finished all three activities, submit your responses below. Module 3 is the final core module; your capstone activity begins immediately after. Your completed operative decision document and prototype from Activity 3 are strong starting material for the capstone.