There are very few – probably none – of our large language model prompts that I am completely happy with. Historically, like pretty much everyone in the industry, we leaned on "prompt engineering". I think this is an unfortunate term: it often feels more like playing "prompt whack-a-mole"

Given the long latency that LLMs currently introduce doing inference, streaming the responses to the user as they are generated is almost a requirement in AI applications. Streaming decreases the perceived and actual wait for results – the user can start reading and reflecting on the results as soon as output

If you use the AWS SDK for JavaScript in e.g. a node.js application, invoking a command on an AWS service generally involves instantiating a client for the service (e.g. DynamoDBClient or S3Client) and invoking a method on that client. The client will do all the work to

The latency to complete inference with a large language model (LLM) such as OpenAI's GPT can be uncomfortably high. Multiply that by needing to make sequentially LLM requests and you have a recipe for a poor user experience. You have probably noticed or read about how ChatGPT for