Почитал на досуге больше о GPT-3 и только сейчас осознал всю крутость принципа ее работы. Примеры примерами, они не сильно поражают когда знаешь, что с хорошим датасетом современные модели можно натренировать генерировать что-угодно достаточно успешно. Но вот GPT-3 никто не тренирует и не fine-tune'ит. Тренировочные примеры это просто часть инпута который скармливается модели при каждом запуске. В общем, почитайте лучше оригинальную статью, там много иллюстраций.
Еще нашел интересный пост с нюансами ее применения и небольшим срывом покровов:
- The API is slow. This is expected: the model has 175 billion parameters; it's too big to fit on a GPU (the largest GPU in the world has 48 GB of VRAM). There is currently no information about the infrastructure running GPT-3.
- The demos you're seeing around the internet are highly biased. The vast majority of output is of significantly lower quality.
- The author trained GPT-3 on his tweets; he estimates that only 30-40% were usable. This implies a 60-70% failure rate.
- If it takes three tries to generate a usable React component, it's probably more practical to just write the code yourself.
- Everyone is using the same model, so there's no competitive advantage.
- With the wrong prompt, you might get racist or sexist output. (Похуй)
- There is still no information on API pricing/cost. (Уже есть)
Еще нашел интересный пост с нюансами ее применения и небольшим срывом покровов:
- The API is slow. This is expected: the model has 175 billion parameters; it's too big to fit on a GPU (the largest GPU in the world has 48 GB of VRAM). There is currently no information about the infrastructure running GPT-3.
- The demos you're seeing around the internet are highly biased. The vast majority of output is of significantly lower quality.
- The author trained GPT-3 on his tweets; he estimates that only 30-40% were usable. This implies a 60-70% failure rate.
- If it takes three tries to generate a usable React component, it's probably more practical to just write the code yourself.
- Everyone is using the same model, so there's no competitive advantage.
- With the wrong prompt, you might get racist or sexist output. (Похуй)
- There is still no information on API pricing/cost. (Уже есть)