This downloads a default pretrained model and tokenizer for Questioning Answering. Now you can use the qa_model on your target question / context:
qa_model( question="Where are the best cookies?", context="The best cookies are in Aporia's office.")# ==> {'score': 0.8362494111061096,# 'start': 24,# 'end': 39,# 'answer': "Aporia's office"}
Extract Embeddings
To extract embeddings from the model, we'll first need to do two things:
Pass output_hidden_states=True to our model params.
When we call pipeline(...) it does a lot of things for us - preprocessing, inference, and postprocessing. We'll need to break all this, so we can interfere in the middle and get embeddings 😉
In other words:
qa_model =pipeline("question-answering", model_kwargs={"output_hidden_states": True})# Preprocessmodel_inputs =next(qa_model.preprocess(QuestionAnsweringPipeline.create_sample( question="Where are the best cookies?", context="The best cookies are in Aporia's office.")))# Inferencemodel_output = qa_model.model(input_ids=model_inputs["input_ids"])# Postprocessingstart, end = model_output[:2]qa_model.postprocess([{"start": start, "end": end, **model_inputs}])# ==> {'score': 0.8362494111061096, 'start': 24, 'end': 39, 'answer': "Aporia's office"}
And finally, to extract embeddings for this prediction:
The next step would be to store your predictions in a data store, including the embeddings themselves. For more information on storing your predictions, please check out the Storing Your Predictions section.
For example, you could use a Parquet file on S3 or a Postgres table that looks like this:
Integrate to Aporia
Now let’s add some monitoring to this model 🚀 To monitor this model in Aporia, the first step is to create a model version: