• @sacredfire@programming.dev
    link
    fedilink
    110 months ago

    Why does a pre-trained model need expensive private hardware after it was trained, other than to handle API requests faster? Is Open AI training chat-GPT on inferior hardware compared to these sophisticated private versions you mentioned?

    • @GBU_28@lemm.ee
      link
      fedilink
      English
      310 months ago

      The fine tuning, while much more efficient than starting fresh, can still be a large amount of work.

      Then consider that your target corpus of data may also be large.

      Then consider to do your reasoning tasks across that corpus also takes strong hardware to get production ready response times.

      No, openai isn’t using inferior hardware, but their model goals, token chunking strategies and overall corpus are generalist in nature.

      There are then processing strategies teams are using to go beyond the “memory” limitations gpt 4 has, that provide massive benefits to coherency, essentially anti hallucination and better overall reasoning