DC Elective 4: Multimodal Composition and Digital Narratives

Digital Communication Certificate · Elective 4

Multimodal Composition and Digital Narratives

This elective treats multimodal composition as a rhetorical and design practice. You will apply the AI literacy framework to AI tools for multimodal work and design an assignment that teaches the multimodal conventions of your discipline. The elective ends with a formatted assignment sheet ready for your course. Plan on 3 to 4 hours: about 2 on the activities and the builder, the rest reading.

What this elective asks you to do

Analyze how the modes and design choices of a disciplinary text serve its purpose. (Activity 1)
Identify the visual conventions of your field a text follows or departs from. (Activity 1, prompt 4)
Analyze a generative AI tool for multimodal work and decide when it belongs in your assignments. (Activity 2)
Design a multimodal assignment with mode requirements, criteria, and process steps. (Assignment builder)

Section 1

What Multimodality Means

Multimodal composition makes meaning by combining semiotic modes: linguistic, visual, aural, spatial, and gestural. A video essay, a data visualization, a podcast with a transcript, an annotated image, and a slide presentation each combine modes to produce meanings no single mode would carry alone. Multimodal literacy is the ability to read and produce these combinations, knowing how the modes interact and what each contributes.

Kress (2010) and the New London Group (1996) start from one claim. No mode is neutral. Each mode has its own affordances and constraints, its own history of use, and its own unstated norms for what counts as credible, appropriate, or persuasive in a given setting. A graph presents quantitative relationships with an appearance of objectivity that prose lacks. A photograph claims the status of evidence in a way an illustration does not. A spoken voice carries the presence of a body. Once students understand these affordances, they have grounds for deciding which modes to use and how.

Conventions differ by discipline

Each discipline sets its own standards for multimodal work. Engineering reports follow visual conventions that differ from those of public health infographics, and both differ from the humanities essay-film. A student who learns multimodal composition as a generic skill, without the conventions of a field, carries little of it into professional practice. This elective helps you teach multimodal composition through the conventions your discipline uses.

Section 2

Design Decisions as Rhetorical Choices

Every design decision in multimodal composition is a rhetorical choice. It makes some meanings available and closes off others. Font, color palette, layout, the relation of image to text, the pacing of a video, and the navigation structure of a website all communicate something to the reader.

When you design a multimodal assignment, state the task with enough rhetorical precision that students know which choices they are expected to make and the criteria you will use to judge them. "Create a video about your research" names a format and stops there. A multimodal assignment specifies the rhetorical situation, the intended audience, the limits on mode selection, and the criteria for judging design choices. Writing those criteria requires knowing what the design choices are meant to achieve.

To teach multimodal composition, you need to name the design choices a format offers, explain why some serve the purpose better than others, and say what evidence of deliberate design you will look for in student work. The activities in this elective build that ability for your discipline.

Section 3

AI Tools in Multimodal Composition: Applying the Literacy Framework

Text-to-image systems, video generators, audio synthesis tools, and AI-assisted design platforms have moved quickly into multimodal composition. Students already use them. Your task is to address them in ways grounded in rhetorical understanding and structural analysis.

The AI literacy framework (Cole, 2026) brings out questions about these tools that generic AI policy language leaves unasked.

At the conceptual level: text-to-image systems are diffusion models trained on image-text pairs scraped from the web. They produce an image by predicting what visual content tends to appear with prompts statistically similar to the one entered. They do no reasoning about what the prompt means. Their output reflects the distribution of images and text in the training data, and that data comes mostly from particular cultural, linguistic, and aesthetic settings. Each generated image is a statistical aggregate of an unrepresentative corpus.

At the structural level: large web-scraped training sets overrepresent some voices and viewpoints, as Bender et al. (2021) argue, and image generators carry the same pattern into pictures. Bianchi et al. (2023) found that ordinary prompts, including prompts that never mention identity, lead text-to-image models to reproduce demographic stereotypes. Luccioni et al. (2023) found that three widely used systems underrepresent marginalized identities. Test a prompt for a "professional," "authoritative," or "neutral" image and look at whose bodies and which aesthetics appear by default. Test a concept from a specific cultural tradition and check the result for accuracy. The hierarchies in the training data become the tool's defaults. Students who use these tools without structural literacy produce visuals whose biases they do not see. Teaching multimodal composition includes teaching that structural analysis, beyond the mechanics of the tool.

At the operative level: under what conditions is using a generative AI tool in a multimodal assignment defensible? Start from what the assignment is for. If it asks students to practice deliberate design choices, a tool that makes those choices for them defeats its purpose. If it asks students to analyze AI-generated visual content critically, using the tools is the point. Your decision about AI use follows from the purpose of the assignment.

Section 4

Multimodal Composition in Your Discipline

Each discipline has its own multimodal conventions. The relation between image and caption in a biology lab report follows different norms from the relation between image and text in an art history essay. Scientific visualization makes claims to truth by different conventions than journalistic photography. Practitioners absorb these conventions over years inside professional communities. Students are new to the discipline's visual rhetoric, so they need the conventions stated outright.

A well-designed multimodal assignment states the conventions: which visual genres the field uses, what rhetorical work they do, which choices convention fixes and which it leaves open, and what doing them well looks like. The artifact at the end of this elective asks you to design such an assignment, drawing on your analyses in Activities 1 and 2.

Activity 1

Multimodal Analysis of a Disciplinary Text

Individual analysis · 45–60 minutes

Select one multimodal text from your discipline: a published visualization, a public explainer video, a conference poster, an interactive data tool, a website, or another example of professional communication in your field. Analyze it as a multimodal composition, attending to its design choices and their rhetorical effects.

What a complete response does

Identifies the text, its audience, and its purpose.
Explains how the modes interact.
Ties three or four design choices to the text’s purpose.
Names conventions the text follows and departs from.

Prompts marked * need at least one full sentence each to submit the elective.

Describe the text you selected. (Format, medium, creator or publication, intended audience, and rhetorical purpose. What is this text trying to do?) Mode analysis. (Which modes are present: linguistic, visual, aural, spatial, gestural? How do they interact? What does the combination achieve that no single mode would achieve alone?) Design choices. (Identify three or four specific design decisions, such as font, layout, color, image selection, pacing, or navigation. For each, name the rhetorical work it does. What meaning does it create, and what alternatives would have produced different meanings?) Disciplinary conventions. (What conventions of your discipline's visual rhetoric does this text follow? What conventions does it deviate from? How would someone outside your discipline likely read it differently than a practitioner?) Pedagogical application. (How would you use this text in your course? As a model to analyze, a professional exemplar, a case study in conventions, or something else? What would you want students to see that they might not see on their own?)

Contribute to the repository

Analyses that identify a field's conventions and their rhetorical functions give faculty in other disciplines vocabulary for teaching multimodal composition.

Yes, share my multimodal analysis with the C²TC repository. CWSP reviews each item before posting it and credits you by name and department unless you ask otherwise.

Activity 2

AI Tool Analysis: Generative AI in Your Multimodal Context

Individual analysis · 30–45 minutes

Select one generative AI tool relevant to multimodal composition in your discipline: a text-to-image system, an AI-assisted design tool, an audio synthesis tool, a video generator, or an AI writing assistant for visual descriptions. Use the three-level AI literacy framework (Cole, 2026) to analyze what the tool does, which hierarchies it reproduces, and what decision about its use in your multimodal assignments you are able to defend.

What a complete response does

Describes the tool and its training data where known.
Names whose conventions the outputs center.
States the conditions under which the tool belongs in your assignments.

Prompts marked * need at least one full sentence each to submit the elective.

Identify and describe the tool. (Name, claimed function, and technical description. What kind of generative AI system is it, and what training data does it use?) Structural analysis. (What hierarchies does this tool reproduce in its outputs? Whose aesthetic conventions, bodies, or cultural frameworks does it center? Who bears the cost of its failures? Which prompts produce inaccurate, stereotyped, or exclusionary outputs?) Operative decision: this tool in your multimodal assignments. (Given your structural analysis, under what conditions is using this tool in your assignments defensible? Which purposes would its use serve, and which would it undermine? Would you permit it, require it, prohibit it, or require critical analysis of its use?)

Contribute to the repository

Decisions about specific generative AI tools, grounded in structural analysis and tied to real assignments in a discipline, help the C²TC community write consistent, defensible AI use policies for multimodal work.

Yes, share my AI tool analysis and operative decision with the C²TC repository. CWSP reviews each item before posting it and credits you by name and department unless you ask otherwise.

Your Artifact

Build Your Multimodal Assignment

Complete the fields below to generate a formatted, print-ready assignment sheet for your course. Draw on your analyses from Activities 1 and 2 in every field. This document brings that work together.

What a complete response does

Specifies format, medium, and rhetorical situation.
Names the required modes and the design choices students make deliberately.
Gives three to five criteria, each describing strong work.
Requires a design rationale or process record.

Prompts marked * need at least one full sentence each to submit the elective.

Assignment title (Specific and descriptive. Choose a title that signals the rhetorical task to students; "Multimodal Project" does not.) Course and audience (Course number and title, and a brief description of the student audience.) Assignment overview (2–4 sentences describing the task and its purpose in the course. Write for students, since this text appears at the top of the assignment sheet.) Task description (Specify what students will produce: format, medium, length or duration, and the rhetorical situation. Who is the audience, what is the purpose, and what constraints apply?) Mode and design requirements (Specify which modes students must include and what design choices they are expected to make deliberately. Draw on the disciplinary conventions from your Activity 1 analysis.) AI use policy for this assignment (Your operative decision from Activity 2, stated as a clear policy for students. Say which AI uses are permitted, which are prohibited, and which students must document and reflect on.) Assessment criteria (3–5 criteria that separate excellent from adequate multimodal composition in this assignment. Write each as a description of quality, in sentences.) Process requirements (What will students submit or document beyond the final product: a design rationale, a process journal, peer review, a draft? These requirements teach students to make multimodal decisions as well as to produce multimodal work.) Resources you will provide (What scaffolding, examples, or tools will you make available? List readings, model texts, tool guides, or peer review structures.)

Reading List

Selected Sources

Cole, K. K. (2026). AI literacy is not one thing: Conceptual, structural, and operative levels for higher education governance. Pedagogies: An International Journal, forthcoming. Framework: cwspwolf.com/ai_literacy_unified.html.
Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. Routledge.
New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60–92.
Selber, S. A. (2004). Multiliteracies for a digital age. Southern Illinois University Press.
Shipka, J. (2011). Toward a composition made whole. University of Pittsburgh Press.
Rodrigue, T. K., et al. (2016). Navigating the soundscape, composing with audio. Kairos, 21(1).
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots. FAccT '21.
Bianchi, F., Kalluri, P., Durmus, E., Ladhak, F., Cheng, M., Nozza, D., Hashimoto, T., Jurafsky, D., Zou, J., & Caliskan, A. (2023). Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency.
Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81.
Luccioni, A. S., Akiki, C., Mitchell, M., & Jernite, Y. (2023). Stable bias: Evaluating societal representations in diffusion models. Advances in Neural Information Processing Systems 36 (Datasets and Benchmarks Track).
Arola, K. L., Sheppard, J., & Ball, C. E. (2014). Writer/Designer: A guide to making multimodal projects. Bedford/St. Martin's.

Where you will use this work

Name the course and term where you will use what you built in this elective, if you know them. (Optional. CWSP sends one question near the end of that term.)

Four questions about this elective

Your answers here go to CWSP without your name or email, apart from your elective responses. CWSP reads them to revise the elective. They are optional, and they send when you submit.

The readings and activities applied to a course I teach. The instructions for each activity were clear. About how many hours did the elective take you? What one change would make this elective more useful for your teaching?

Complete Elective 4

When you have finished both activities and built your artifact, submit your responses. Your work is auto-saved in your browser.

Submitted. A copy of your responses comes to your NC State email within a few minutes. If it has not arrived within an hour, email kkcole2@ncsu.edu. Your responses also stay saved in this browser.

Submission problem. Your responses are saved locally. Try again or email kkcole2@ncsu.edu.

Return to

← Digital Communication Certificate