Welcome to AI Office Hours!

Welcome to the very first AI Office Hours Newsletter! I'm Sinan Ozdemir, your guide through the ever-evolving world of AI. As a former lecturer at the Johns Hopkins University and an experienced entrepreneur in the AI field, I've spent years breaking down complex concepts, building real-world solutions, and sharing my knowledge through various publications. Now, I'm thrilled to welcome you to this journey where we demystify and actually use AI, particularly the realm of Large Language Models (LLMs).

Am I an experience blogger or newsletter writer? Nope. Do I care a lot about sharing actionable insights and code for my fellow software engineers on the topic of AI? Absolutely!

Example 1: Generating text with Open-source FLAN-T5

Our first example today is a simple one, but something that I get asked about a fair amount: How do I simply generate text from an open source model from Huggingface?

Let's take the example of Google's FLAN-T5 model, one of Google’s latest open-sourced LLM. Using FLAN-T5 - which is a sequence to sequence model which matters for our upcoming code - we can generate a piece of text based on a given prompt. Here's a quick Python code snippet using the transformers library from Hugging Face:

# Import necessary classes from the transformers library
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

# Define the model we want to use
MODEL = "google/flan-t5-base"

# Initialize the tokenizer using the from_pretrained method 
tokenizer = AutoTokenizer.from_pretrained(MODEL)

# Initialize the model using the from_pretrained method
model = AutoModelForSeq2SeqLM.from_pretrained(MODEL)

# Define our prompt text
prompt = "Translate from English to Spanish: 'How are you?'"

# Encode our prompt text into tensor of integers representing the sequence of tokens
inputs = tokenizer.encode(prompt, return_tensors='pt') 

# Generate the output sequence using the model
outputs = model.generate(inputs, max_length=100) 

# Decode the output sequence into readable text
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)

# Print the generated text
print(generated_text)  # outputs "Cómo estás?"

1. Model and Tokenizer Initialization: The necessary classes are imported from the transformers library and the pre-trained model (in this case, the FLAN model) is specified. Then, both the tokenizer and the model are initialized based on the pre-trained model.

2. Prompt Definition: The input text, or prompt, is defined. This is the text that the model will translate or generate text from.

3. Input Preparation: The prompt is encoded into a sequence of tokens (a format that the model can understand) using the tokenizer. This involves converting the text into a tensor of token IDs.

4. Text Generation: The model generates an output sequence based on the input tensor. The length of the output sequence is controlled by specifying a maximum length with the max_length parameter.

5. Output Decoding: The output sequence is decoded back into readable text using the tokenizer. Special tokens included in the output sequence are removed during this process.

6. Printing the Output: The final step involves printing the generated text. Depending on the task, this could be a translation, a summary, a continuation of the prompt, or any other type of text.

This is a pretty bare bones code example but I didn’t want to leave you totally hanging on the first post 🙂.

Next time on AI Office Hours

In the coming weeks, expect more content around prompting techniques, using and fine-tuning open source LLMs, using and testing different closed source LLMs all with a mind for production and keeping costs down and solving interesting and specific tasks with LLMs. This is something I love talking about and building around, so I can’t wait 🙂 

I encourage you to be curious, ask questions that I can talk about on the newsletter, and experiment with all of these examples. After all, AI is as much about learning and adapting as it is about coding and algorithms.

There’s also a github I’ll do my best to maintain with any code examples here: