璃香小代

I once used GPT-4 to generate prompts about making Anki flashcards, inspired by auto prompts, aimed at having AI continuously adjust its prompts after evaluating the quality of output results to achieve optimal performance; experiments on Claude and GPT-4 with the following prompt:

When creating flashcards, please fully reference SuperMemo's 20 rules, questions from AP exams and other tests, and guiding questions from various tutorials. At the same time, pay attention to the following requirements:

1. Ensure that flashcards are concise, clear, and focused on key information.
2. Questions should be specific and clear, avoiding ambiguity.
3. Use simple and direct language to ensure the cards are easy to read and understand.
4. Answers should contain only one key fact/name/concept/term.
5. Consider the applicability and universality of the questions, ensuring they have practical value in different contexts and knowledge areas.
6. Pay attention to the reasonableness of the questions, ensuring they do not cause misunderstanding or doubt.
   Here is an overview of SuperMemo's 20 rules:
7. Do not learn if you do not understand.
8. Learning materials should be in the form of questions and answers.
9. Minimum information principle: each question should be as concise as possible.
10. Avoid Boolean questions (yes/no questions).
11. Optimize the phrasing of questions.
12. Use concrete rather than abstract questions.
13. Focus on applicable knowledge rather than pure theoretical knowledge.
14. Balance the difficulty of questions, avoiding questions that are too simple or too complex.
15. Use diverse question formats to increase learning interest.
16. Ensure answers are easy to recall and understand.
17. Use contextual clues to aid recall.
18. Consider multiple learning perspectives when creating questions.
19. Break knowledge down into independent, manageable parts.
20. Use images, charts, and other visual aids when appropriate.
21. Develop an appropriate review plan to ensure long-term memory.
22. Use spaced repetition to enhance memory retention.
23. Create specific questions for easily confused knowledge points.
24. Integrate knowledge from different fields to form a higher-level understanding.
25. Regularly update and modify flashcards to ensure the accuracy and timeliness of knowledge.
26. Provide feedback and adjustments to your learning process to improve learning effectiveness.
    Text: The sun is the only star in the solar system, composed of hydrogen and helium. The core temperature of the sun is about 15 million degrees Celsius. At such high temperatures, hydrogen nuclei undergo fusion reactions, forming helium atoms and releasing a large amount of energy.

According to SuperMemo's 20 rules, we can extract information from the text and create the following flashcards:

Text: The time it takes for the Earth to complete one rotation is 23 hours, 56 minutes, and 4 seconds, known as a sidereal day. The time it takes for the Earth to orbit the sun once is about 365.24 days, known as a tropical year.
Create flashcards according to SuperMemo's 20 rules:

This set of flashcards follows the following rules:

• Rule 2: Learning materials are based on questions and answers.
• Rule 3: Minimum information principle, questions are as concise as possible.
• Rule 5: Optimize the phrasing of questions to make them clear and understandable.
• Rule 10: Ensure answers are easy to recall and understand.
• Rule 13: Break knowledge down into independent, manageable parts.
  Through these flashcards, we can see how to apply SuperMemo's 20 rules in the actual creation process. Of course, depending on the content of the text and the learning objectives, other rules may need to be adopted. The key is to comprehensively consider these rules when creating flashcards to enhance learning effectiveness.
  When creating flashcards, please pay attention to the following points:

1. Flexibly apply SuperMemo's 20 rules for different knowledge areas and backgrounds.
2. Combine practical needs, reference questions from AP exams and other tests, as well as guiding questions from various tutorials.
3. Pay attention to the applicability and universality of the questions, ensuring they have practical value in different contexts and knowledge areas.
4. Maintain the reasonableness of the questions to avoid causing misunderstanding or doubt.
   By integrating the above points, we can create flashcards more effectively and enhance learning outcomes. Here is the text I want to provide:
   。。。。。。。。

Example#

A phenomenon was observed on Claude and GPT-4, where nearly 80% of the Q&A generated from texts that illustrate a detail through numerous examples could not exist independently from the text and required manual modifications;

Detail Fact Decomposition#

GPT-4 can propose independent questions well, but when answers involve multiple steps or points, it easily falls into a listing nightmare, neglecting the decomposition of points and steps to form more memorable cards. Its understanding of facts is key from the perspective of the question, rather than the answer.

Claude not only has a small amount of mixed Chinese and English phenomena but also exhibits a chain question phenomenon, equivalent to having three sub-questions related to the main question under one Q&A, overly abbreviated, such as "What does this reflect?" "It reflects the consistency between formulas."

Retaining LaTeX Formulas#

The LaTeX formulas I used are recognized by the free OCR from Haowei in Quicker, so at least I don't have to consider the Mathpix payment issue. After completing the text OCR, I then recognize the formula parts one by one.

Selection of Learning Materials#

Good introductory textbooks allow beginners to get started faster, just like the "Advanced Algebra" published by Fudan University. The more concise and logical the text statements are, the better the generated Q&A will be. However, the downside is that the ability to transform questions into different forms with the same logic and thought is relatively poor. The writing style of the input text must be consistent; otherwise, the Waluigi effect can easily increase.

Length Issue#

We can see that content detailing statements often generates more detailed Q&A, and as the length of the text and dialogue increases, the Waluigi effect also increases.

Model Issues#

Claude, while having a small amount of mixed Chinese and English phenomena, maintains strong consistency, but feedback debugging is relatively poor, and the redundancy of questions remains good. GPT-4 tends to generate overly general questions but has stronger background provision capabilities than Claude, with better feedback debugging ability, but has too few questions and poor redundancy.

Prompt Issues#

When writing prompts, it is important to focus on patterns and specificity, concentrating on what to do rather than what not to do.

Based on Claude, GPT-4 generates content targeting dictionary texts that easily leads to "give an example" questions (also fewer in quantity), because there are no optimized questions for dictionary texts. There is also a fine understanding issue; they cannot ensure superiority in deep literary questions. Simply put, it is the language subdivision knowledge of ancient people's "elementary school." They generate high-quality, correct content based on the frequency of language and domain knowledge included in the original training data, but the probability of generating incorrect or ambiguous usages still exists. They will automatically correct to conform to incorrect or ambiguous but seemingly correct usages.
I once wrote a prompt about generating issues related to dictionaries and textual explanations, which differs greatly from the above prompt, such as:

Sentence miner in language learning is a kind of people who would use the grammar books or dictionaries to create flashcards (mostly in Q&A forms)
For example, text: welcome2 ●●● S2 W3 adjective
1 you’re welcome SPOKEN a polite way of replying to someone who has just thanked you for something
‘Thanks for the coffee.’ ‘You’re welcome.’
2 if someone is welcome in a place, other people are glad that they are there
I had the feeling I wasn’t really welcome.
I didn’t feel welcome in the club.
Mary made us very welcome.
We try to make the new students feel welcome.
3 if something is welcome, you enjoy it because you feel that you need it
The weekend was a welcome break from the pressures of work.
Six months in Scotland would make a welcome change from London.
A cup of tea would be very welcome.
4 if something is welcome, you are glad that it has happened
The increase in interest rates is welcome news for investors.
This new funding will come as a welcome boost for the industry.
5 be welcome to something SPOKEN used to say that someone can have something if they want it, because you certainly do not want it
If you want to take the job you’re welcome to it!
6 be welcome to do something SPOKEN used to invite someone to do something if they would like to
You’re welcome to stay for lunch.
Q&A: Q: How might someone respond with "you're welcome"? (SPOKEN)
A: "You're welcome" is a polite way of replying to someone who has just thanked you for something.
Q: In what situations might someone not feel welcome?
A: If someone is not welcome in a place, it means that other people are not glad that they are there.
Q: What is welcome's meaning in "Mary made us very welcome"?
A: If someone is welcome in a place, other people are glad that they are there. So it means Mary made other people glad that we are there.
Q: What is welcome's meaning in "The weekend was a welcome break from the pressures of work."?
A: if something is welcome, you enjoy it because you feel that you need it. So it means the break is what I need.
Q: When might someone find something welcome?
A: If something is welcome, it means you enjoy it because you feel that you need it.
Q's the meaning of welcome in "The increase in interest rates is welcome news for investors"?
A: if something is welcome, you are glad that it has happened, so it means the increase in interest rates has happened.
Q: How might someone use "you're welcome" to invite someone to do something?
A: "You're welcome to stay for lunch".
Q: What's the meaning of "be welcome to something"(SPOKEN)?
A: It is used to say that someone can have something if they want it, because you certainly do not want it.
Q: What's the meaning of "be welcome to something" in "If you want to take the job you’re welcome to it!"
A: It means I can take the job because the speaker certainly does not want it.
Q: What's the meaning of "be welcome to do something"(SPOKEN)?
A is used to invite someone to do something if they would like to.
Q: What's the meaning of "be welcome to do something" in "You’re welcome to stay for lunch."
A: It is used to invite the listener to stay for lunch.
I will provide you the text to generate the Q&A forms.

Logical Thinking (CoT)#

Using AI to form Q&A has another flaw; they often target a single grammatical part of the sentence for questioning, but forming Q&A should not only include "What is the relationship between xxx and xxx?" "yyy is the relationship between xxx and xxx." It should specifically include induction, comparison, and the logical flow of the text (using fluent and logical language helps in constructing knowledge and language templates; simplified expressions can sometimes lack rigor, and templates can be further decomposed, such as in mathematics or computer science thinking, continuously refining, and then learning to mobilize different atomic knowledge modules and atomic method steps for complex problems).

Insights:#

• Avoid true/false questions: If a Q&A only requires a judgment of right or wrong without providing reasons, it can easily confuse and frustrate learners.

• Background and references: To make Q&A more independent of the text, we need to provide clear background and references, letting learners know which academic field the knowledge belongs to. For example, a function in mathematics is a type of mapping relationship, while in computer science, it is another term for a method. Similar to literature citations, we can introduce information sources for future reference and modification.

• Diversity and redundancy of questions: To give learners a deeper understanding of a detail, we need to ask questions from different angles, stimulating their active recall and thinking abilities. Therefore, try to generate multiple quality questions. These questions will approach the topic from different angles, examining understanding at different levels. For example, for a concept, we can simultaneously inquire about its definition, function, properties, etc.

• Prompts and cards need to be continuously modified; in this article, I will record different versions of prompts and their effects.

As follows:

Generate flashcards based on text. (Tentative)
Flashcards are a powerful learning tool. They’re also a pain in the butt to make.
Some readers said they were using ChatGPT to generate flashcards for subjects they’re studying. This seems well within the LLM abilities as a “calculator for words.” Thus, with the correct prompts, you could get fairly good results here—provided you’re inputting the material you wish to see transformed into flashcards and not expecting the LLM to get the facts on its own (see below).
However, given the difficulty of making “good” flashcards, I wouldn’t enter any into my Anki without reviewing them first. Nonetheless, making flashcards is tedious, so getting a first draft that I later review might speed up the process considerably. The risks seem relatively limited if you confirm the cards’ correctness before putting them in your deck.——Scott.H.Young