Increase max input length for HuggingFace model in SageMaker deployment

--

I deployed HuggingFace zephyr-7b-beta model to SageMaker by using the default deploy.py script. When trying to invoke the model endpoint, I received the error “ValueError: Error raised by inference endpoint: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (422) from primary with message “{“error”:”Input validation error

--

--

Jackie Chen
Jackie Chen’s IT Workshop

We are all apprentices in a craft where no one ever becomes a master.