AI Test Bed: OpenStax Textbooks (Creative Commons Attribution 4.0 International License)
Source: https://openstax.org/
OpenStax’s mission is to improve educational access and learning for everyone. OpenStax is part of Rice University, which is a 501(c)(3) nonprofit.
Advanced Placement® and AP® are trademarks registered and/or owned by the College Board, which is not affiliated with and does not endorse this site. Give today and help us reach more students.
Test Lab: https://labs.perplexity.ai/
- Algebra and Trigonometry 2e:
https://ideal-group.org/ai-library/Algebra-and-Trigonometry-2e.pdf - American Government 3e:
https://ideal-group.org/ai-library/AmericanGovernment3e.pdf - AP Biology:
https://ideal-group.org/ai-library/APBiology.pdf - Biology 2e:
https://ideal-group.org/ai-library/Biology2e.pdf - Business Ethics:
https://ideal-group.org/ai-library/BusinessEthics.pdf - Business Law I Essentials:
https://ideal-group.org/ai-library/Business_Law_I_Essentials.pdf - Business Statistics:
https://ideal-group.org/ai-library/IntroductoryBusinessStatistics.pdf - Calculus Volume 1:
https://ideal-group.org/ai-library/Calculus_Volume_1.pdf - Calculus Volume 2:
https://ideal-group.org/ai-library/CalculusVolume2.pdf - Calculus Volume 3:
https://ideal-group.org/ai-library/CalculusVolume3.pdf - College Algebra 2e:
https://ideal-group.org/ai-library/College-Algebra-2e.pdf - College Algebra Corequisite Support 2e:
https://ideal-group.org/ai-library/College-Algebra-Corequisite-Support-2e.pdf - Concepts of Biology:
https://ideal-group.org/ai-library/ConceptsofBiology.pdf - Contemporary Mathematics:
https://ideal-group.org/ai-library/ContemporaryMathematics.pdf - Economics 3e:
https://ideal-group.org/ai-library/Economics3e.pdf - Elementary Algebra 2e:
https://ideal-group.org/ai-library/ElementaryAlgebra2e.pdf - Entrepreneurship:
https://ideal-group.org/ai-library/Entrepreneurship.pdf - Intermediate Algebra 2e:
https://ideal-group.org/ai-library/IntermediateAlgebra2e.pdf - Introduction to Anthropology:
https://ideal-group.org/ai-library/IntroductiontoAnthropology.pdf - Introduction To Business:
https://ideal-group.org/ai-library/IntroductionToBusiness.pdf - Introduction to Intellectual Property:
https://ideal-group.org/ai-library/Introduction_to_Intellectual_Property.pdf - Introduction to Philosophy:
https://ideal-group.org/ai-library/Introduction_to_Philosophy.pdf - Introduction to Political Science:
https://ideal-group.org/ai-library/Introduction_to_Political_Science.pdf - Introduction to Sociology 3e:
https://ideal-group.org/ai-library/IntroductiontoSociology3e.pdf - Introductory Business Statistics 2e:
https://ideal-group.org/ai-library/Introductory_Business_Statistics_2e.pdf - Introductory Statistics 2e:
https://ideal-group.org/ai-library/Introductory_Statistics_2e.pdf - Introductory Statistics:
https://ideal-group.org/ai-library/IntroductoryStatistics.pdf - Macroeconomics 3e:
https://ideal-group.org/ai-library/Macroeconomics3e.pdf - Microeconomics 3e:
https://ideal-group.org/ai-library/Microeconomics3e.pdf - Nutrition for Nurses:
https://ideal-group.org/ai-library/Nutrition_for_Nurses.pdf - Prealgebra 2e:
https://ideal-group.org/ai-library/Prealgebra2e.pdf - Precalculus 2e:
https://ideal-group.org/ai-library/Precalculus_2e.pdf - Principles of Finance:
https://ideal-group.org/ai-library/PrinciplesofFinance.pdf - Principles of Marketing:
https://ideal-group.org/ai-library/Principles_Marketing.pdf - Psychology 2e:
https://ideal-group.org/ai-library/Psychology2e.pdf - Statistics:
https://ideal-group.org/ai-library/Statistics.pdf - U.S. History:
https://ideal-group.org/ai-library/USHistory.pdf - World History Volume 1:
https://ideal-group.org/ai-library/World_History_Volume_1.pdf - World History Volume 2:
https://ideal-group.org/ai-library/World_History_Volume_2.pdf
Llama 3 Examples:
Where possible, we try to match the Hugging Face implementation.
Model | Parameter Count | Context Length | Model Type |
---|---|---|---|
llama-3-8b-instruct |
8B | 8192 | Chat Completion |
llama-3-70b-instruct |
70B | 8192 | Chat Completion |
codellama-70b-instruct |
70B | 16384 | Chat Completion |
mistral-7b-instruct [1] |
7B | 16384 | Chat Completion |
mixtral-8x7b-instruct |
8x7B | 16384 | Chat Completion |
mixtral-8x22b-instruct |
8x22B | 16384 | Chat Completion |
[1] This model refers to the
v0.2
release ofmistral-7b-instruct
.
The parameters MMLU, GPQA, MATH, and HumanEval stand for the following in benchmarking:
- MMLU: Meaningful Machine Learning Understanding. It is a benchmark that measures a model’s understanding of machine learning concepts.
- GPQA: General Purpose Question Answering. It is a benchmark that evaluates a model’s ability to answer general-purpose questions.
- MATH: Mathematical Reasoning. It is a benchmark that measures a model’s ability to reason mathematically.
- HumanEval: Human Evaluation. It is a benchmark that involves human evaluators to assess a model’s performance in various tasks.
Parameters | Llama 3 8B | Llama 3 70B | GPT-4 |
MMLU | 68.40 | 82.00 | 86.50 |
GPQA | 34.20 | 39.50 | 49.10 |
MATH | 30.00 | 50.40 | 72.20 |
HumanEval | 62.20 | 81.70 | 87.60 |
DROP | 58.40 | 79.70 | 85.40 |
Huggingface Models: https://huggingface.co/models
Huggingface Course: https://huggingface.co/learn/nlp-course/chapter1/1
Meta Llama4 Community License Agreement: https://huggingface.co/meta-llama/Meta-Llama-3-8B
Meta Llama 3 Version Release Date: April 18, 2024
Pricing: https://huggingface.co/pricing
Spaces Hardware: https://huggingface.co/pricing#spaces
Building something cool as a side project? We also offer community GPU grants.
Community GPU Grants: Do you have an awesome Space but need help covering the GPU hardware upgrade costs? We love helping out those with an innovative Space so please feel free to apply for a community GPU grant and see if yours makes the cut! This application can be found in your Space hardware repo settings in the lower left corner under “sleep time settings”: