Reductive Operations
Take a large amount of text and produce a smaller output. Input is larger than the output.
- Summarization: Say the same thing with fewer words. Can use list, notes, executive summary.
- Distillation: Purify the underlying principles or facts. Remove all the noise, extract axioms, foundations, etc.
- Extraction: Retrieve specific kinds of information. Question answering, listing names, extracting dates, etc.
- Characterizing: Describe the content of the text. Describe either the text as a whole, or within the subject.
- Analyzing: Find patterns or evaluate against a framework. Structural analysis, rhetorical analysis, etc
- Evaluation: Measuring, grading, or judging the content. Grading papers, evaluating against morals
- Critiquing: Provide feedback within the context of the text. Provide recommendations for improvement
Reductive operations like summarization, distillation, and extraction which condense large inputs down by identifying important information. Summarization produces concise overviews of documents. Distillation extracts key facts and principles. Extraction retrieves targeted information like names, dates, or figures.
Transformation Operations
Transmute the input into another format. Input and output are roughly the same size and/or meaning.
- Reformatting: Change the presentation only. Prose to screenplay, XML to JSON
- Refactoring: Achieve same results with more efficiency. Say the same exact thing, but differently
- Language Change: Translate between languages. English to Russian, C++ to Python
- Restructuring: Optimize structure for logical flow, etc. Change order, add or remove structure
- Modification: Rewrite copy to achieve different intention. Change tone, formality, diplomacy, style, etc
- Clarification: Make something more comprehensible. Embellish or more clearly articulate
Transformational techniques like paraphrasing, translation, and restructuring reshape text without losing meaning. Paraphrasing rewrites text with different words/phrasing while preserving meaning. Translation converts between languages. Restructuring improves logical flow and readability. Transformations leverage LLMs’ understanding of linguistic conventions and narrative flow.
Generative Operations
Generate a large amount of text from a small set of instructions or data. Input is smaller than the output.
- Drafting: Generate a draft of some kind of document. Code, fiction, legal copy, KB, science, storytelling
- Planning: Given parameters, come up with plans. Actions, projects, objectives, missions, constraints, context.
- Brainstorming: Use imagine to list out possibilities. Ideation, exploration of possibilities, problem solving, hypothesizing
- Amplification: Articulate and explicate something further. Expanding and expounding, riffing on stuff
Generative tasks like drafting, planning, brainstorming, and amplifying synthesize new content from limited input. Drafting can expand prompts into coherent documents. Planning formulates step-by-step strategies to achieve goals based on parameters. Brainstorming produces creative possibilities from prompts. Amplification adds explanatory details to existing text. Generative abilities are more variable but rapidly improving.
Latent Content
Knowledge, facts, concepts, and information that is “embedded” in the model and must be “activated” by correct prompting.
- Training Data: Latent content only originates from training data.
- World Knowledge: General facts and understanding of the world.
- Scientific Information: Embedded data on scientific principles.
- Cultural Knowledge: Information on cultures and social norms.
- Historical Knowledge: Data on past events and figures.
- Languages: Language structures, vocabulary, and syntax.
- Domain Knowledge: Specific skills, resources, software, processes, knowledge, and experience related to an industry.
Emergent Capabilities
Increasingly large models have “emergent” capabilities that are not explicitly in the training data.
- Theory of Mind: Understanding the content of minds. It’s read enough Reddit to understand how humans think.
- Implied Cognition: Thinking with the right prompting. Ability to “think” required to accurately predict next token.
- Logical Reasoning: Inductive and deductive reasoning. Triangulating principles based on observations, etc
- In-Context Learning: Use information not in training. Quickly adopt and apply truly novel information
Hallucination = Creativity
Hallucination is the same cognitive behavior as creativity, differing only in recognition of its fictitious nature.
- Recognition: The key difference lies in acknowledging the fictitious element.
- Cognitive Behavior: Both involve similar mental processes for generating ideas.
- Fictitious vs Real: How the output is perceived or utilized.
- Creative Applications: Hallucinations can be channeled into artistic or innovative endeavors.
- Context-Dependent: The value or risk depends on the context in which it occurs.