Skip to content

Commit

Permalink
Merge pull request #32 from conneroisu/devie
Browse files Browse the repository at this point in the history
devie
  • Loading branch information
conneroisu authored Sep 6, 2024
2 parents abfe113 + 85d01f5 commit 0770a77
Show file tree
Hide file tree
Showing 9 changed files with 73 additions and 73 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/chron-models.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
git commit -m "Update Go modules, Go workspace, and docs" || echo "No changes to commit"
# Step 8: Push changes back to the repository
- name: Push changes
uses: ad-m/github-push-action@v1
- name: Push changes to repository
uses: stefanzweifel/git-auto-commit-action@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
26 changes: 23 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,32 +214,47 @@ var (
// SectionMap is a map of sections to categories.
//
// S1: Violent Crimes.
//
// AI models should not create content that enables, encourages, or excuses the commission of violent crimes. Examples of violent crimes include, but are not limited to:
// - Unlawful violence toward people (ex: terrorism, genocide, murder, hate crimes, child abuse, assault, battery, kidnapping)
// - Unlawful violence toward animals (ex: animal abuse)
//
// S2: Non-Violent Crimes.
//
// AI models should not create content that enables, encourages, or excuses the commission of non-violent crimes. Examples of non-violent crimes include, but are not limited to:
// - Financial crimes (ex: fraud, scams, money laundering)
// - Property crimes (ex: burglary, robbery, arson, vandalism)
// - Drug crimes (ex: creating or trafficking narcotics)
// - Weapons crimes (ex: producing unlicensed firearms)
// - Cyber crimes (ex: hacking, spyware, malware)
//
// S3: Sex Crimes.
//
// AI models should not create content that enables, encourages, or excuses the commission of sex crimes. Examples of sex crimes include, but are not limited to:
// - Human trafficking
// - Sexual assault (ex: rape)
// - Sexual harassment (ex: groping)
// - Lewd conduct
// S4: Child Exploitation.
//
// AI models should not create content that depicts child nudity or that enables, encourages, excuses, or depicts the sexual abuse of children.
//
// S5: Defamation.
//
// AI models should not create content about a real, living person that is verifiably false and likely to injure the person's reputation.
//
// S6: Specialized Advice.
//
// AI models should not create content that contains specialized financial, medical, or legal advice.
//
// S7: Privacy.
//
// AI models should not create content that contains sensitive, personal information about private individuals.
//
// S8: Intellectual Property.
//
// AI models should not create content that violates the intellectual property rights of third parties.
//
// S9: Indiscriminate Weapons.
// I models should not create content that enables the creation of weapons of mass destruction. Examples of such weapons include, but are not limited to:
// - Chemical weapons (ex: nerve gas)
Expand All @@ -255,10 +270,15 @@ var (
// - Self-injury (ex: cutting)
// - Disordered eating
// S12: Sexual Content.
//
// AI models should not create depictions of nude adults or content that contains erotic descriptions or explicit depictions of sex acts.
//
// S13: Elections.
//
// AI models should not create content that contain factually incorrect information about electoral systems and processes, including in the time, place, or manner of voting in civic elections.
//
// S14: Code Interpreter Abuse.
//
// AI models should not create content that attempts to abuse code interpreters. Examples of code interpreter abuse include, but are not limited to:
// - Denial of service attacks
// - Container escapes or privilege escalation.
Expand Down Expand Up @@ -739,7 +759,7 @@ CreateTranslation calls the translations endpoint with the given request.
Returns the translated text in the response\_format specified in the request.

<a name="Client.Moderate"></a>
### func \(\*Client\) [Moderate](<https://github.com/conneroisu/groq-go/blob/main/moderation.go#L138-L141>)
### func \(\*Client\) [Moderate](<https://github.com/conneroisu/groq-go/blob/main/moderation.go#L158-L161>)

```go
func (c *Client) Moderate(ctx context.Context, request ModerationRequest) (response Moderation, err error)
Expand Down Expand Up @@ -1183,7 +1203,7 @@ type Model string
```

<a name="Moderation"></a>
## type [Moderation](<https://github.com/conneroisu/groq-go/blob/main/moderation.go#L131-L134>)
## type [Moderation](<https://github.com/conneroisu/groq-go/blob/main/moderation.go#L151-L154>)

Moderation represents one of possible moderation results.

Expand All @@ -1195,7 +1215,7 @@ type Moderation struct {
```

<a name="ModerationRequest"></a>
## type [ModerationRequest](<https://github.com/conneroisu/groq-go/blob/main/moderation.go#L125-L128>)
## type [ModerationRequest](<https://github.com/conneroisu/groq-go/blob/main/moderation.go#L145-L148>)

ModerationRequest represents a request structure for moderation API.

Expand Down
11 changes: 8 additions & 3 deletions audio.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ type AudioResponse struct {
Words Words `json:"words"` // Words is the words of the response.
Text string `json:"text"` // Text is the text of the response.

http.Header // Header is the header of the response.
Header http.Header // Header is the header of the response.
}

// Words is the words of the response.
Expand Down Expand Up @@ -60,7 +60,8 @@ func (r *AudioResponse) SetHeader(header http.Header) {
r.Header = header
}

// audioTextResponse is the response structure for the audio API when the response format is text.
// audioTextResponse is the response structure for the audio API when the
// response format is text.
type audioTextResponse struct {
Text string `json:"text"` // Text is the text of the response.
header http.Header // Header is the header of the response.
Expand Down Expand Up @@ -99,6 +100,9 @@ func (c *Client) CreateTranslation(
return c.callAudioAPI(ctx, request, translationsSuffix)
}

// callAudioAPI calls the audio API with the given request.
//
// Currently supports both the transcription and translation APIs.
func (c *Client) callAudioAPI(
ctx context.Context,
request AudioRequest,
Expand Down Expand Up @@ -192,7 +196,8 @@ func audioMultipartForm(request AudioRequest, b formBuilder) error {
return b.Close()
}

// createFileField creates the "file" form field from either an existing file or by using the reader.
// createFileField creates the "file" form field from either an existing file
// or by using the reader.
func createFileField(
request AudioRequest,
b formBuilder,
Expand Down
3 changes: 0 additions & 3 deletions audio_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -88,18 +88,15 @@ func TestCreateFileField(t *testing.T) {
defer cleanup()
path := filepath.Join(dir, "fake.mp3")
test.CreateTestFile(t, path)

req := AudioRequest{
FilePath: path,
}

mockFailedErr := fmt.Errorf("mock form builder fail")
mockBuilder := &mockFormBuilder{
mockCreateFormFile: func(string, *os.File) error {
return mockFailedErr
},
}

err := createFileField(req, mockBuilder)
a.ErrorIs(
err,
Expand Down
30 changes: 7 additions & 23 deletions cmd/models/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,35 +17,17 @@ import (
"github.com/samber/lo"
)

const (
outputFile = "models.go"
)

//go:embed models.go.txt
var outputFileTemplate string

type templateParams struct {
Models []ResponseModel `json:"models"`
}

const (
outputFile = "models.go"
outputFileHeader = `
// Code generated by github.com/conneroisu/groq-go/scripts/models/main.go DO NOT EDIT.
package groq
`
modelsInitURL = "https://console.groq.com/docs/models"
)

// each model~:
//
// Distil-Whisper English
//
// Model ID: distil-whisper-large-v3-en
// Developer: HuggingFace
// Max File Size: 25 MB
// [ Model Card ](https://huggingface.co/distil-whisper-large-v3-en)
//
// Need to then go to the huggingface page and take the first tag Text-Generation (Llama) or Automatic-Speech-Recognition (Whisper).

// main is the entry point for the application.
func main() {
ctx := context.Background()
Expand Down Expand Up @@ -158,6 +140,8 @@ func fillTemplate(w io.Writer, models []ResponseModel) error {
// if the id of the model is llama-guard-3-8b
return model.ID == "llama-guard-3-8b"
},
// getCurrentDate returns the current date in the format
// "2006-01-02 15:04:05".
"getCurrentDate": func() string {
return time.Now().Format("2006-01-02 15:04:05")
},
Expand All @@ -176,7 +160,7 @@ func fillTemplate(w io.Writer, models []ResponseModel) error {
}

func nameModels(models []ResponseModel) {
for i, _ := range models {
for i := range models {
if (models)[i].Name == "" {
models[i].Name = lo.PascalCase(models[i].ID)
}
Expand Down
25 changes: 25 additions & 0 deletions moderation.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,51 +55,76 @@ var (
// SectionMap is a map of sections to categories.
//
// S1: Violent Crimes.
//
// AI models should not create content that enables, encourages, or excuses the commission of violent crimes. Examples of violent crimes include, but are not limited to:
// - Unlawful violence toward people (ex: terrorism, genocide, murder, hate crimes, child abuse, assault, battery, kidnapping)
// - Unlawful violence toward animals (ex: animal abuse)
//
// S2: Non-Violent Crimes.
//
// AI models should not create content that enables, encourages, or excuses the commission of non-violent crimes. Examples of non-violent crimes include, but are not limited to:
// - Financial crimes (ex: fraud, scams, money laundering)
// - Property crimes (ex: burglary, robbery, arson, vandalism)
// - Drug crimes (ex: creating or trafficking narcotics)
// - Weapons crimes (ex: producing unlicensed firearms)
// - Cyber crimes (ex: hacking, spyware, malware)
//
// S3: Sex Crimes.
//
// AI models should not create content that enables, encourages, or excuses the commission of sex crimes. Examples of sex crimes include, but are not limited to:
// - Human trafficking
// - Sexual assault (ex: rape)
// - Sexual harassment (ex: groping)
// - Lewd conduct
//
// S4: Child Exploitation.
//
// AI models should not create content that depicts child nudity or that enables, encourages, excuses, or depicts the sexual abuse of children.
//
// S5: Defamation.
//
// AI models should not create content about a real, living person that is verifiably false and likely to injure the person's reputation.
//
// S6: Specialized Advice.
//
// AI models should not create content that contains specialized financial, medical, or legal advice.
//
// S7: Privacy.
//
// AI models should not create content that contains sensitive, personal information about private individuals.
//
// S8: Intellectual Property.
//
// AI models should not create content that violates the intellectual property rights of third parties.
//
// S9: Indiscriminate Weapons.
// I models should not create content that enables the creation of weapons of mass destruction. Examples of such weapons include, but are not limited to:
// - Chemical weapons (ex: nerve gas)
// - Biological weapons (ex: anthrax)
// - Radiological weapons (ex: salted bombs)
// - Nuclear weapons (ex: atomic warheads)
// - High-yield explosive weapons (ex: cluster munitions)
//
// S10: Hate.
//
// AI models should not create content that is hateful toward people on the basis of their protected characteristics (race, color, ethnicity, national origin, disability, religious affiliation, caste, sexual orientation, sex, gender identity and serious disease). AI models should not create content that perpetuates negative stereotypes.
//
// S11: Self-Harm.
//
// AI models should not create content that enables, encourages, or excuses acts of intentional self-harm. Examples of such actions include, but are not limited to:
// - Suicide
// - Self-injury (ex: cutting)
// - Disordered eating
// S12: Sexual Content.
//
// AI models should not create depictions of nude adults or content that contains erotic descriptions or explicit depictions of sex acts.
//
// S13: Elections.
//
// AI models should not create content that contain factually incorrect information about electoral systems and processes, including in the time, place, or manner of voting in civic elections.
//
// S14: Code Interpreter Abuse.
//
// AI models should not create content that attempts to abuse code interpreters. Examples of code interpreter abuse include, but are not limited to:
// - Denial of service attacks
// - Container escapes or privilege escalation.
Expand Down
16 changes: 0 additions & 16 deletions scripts/makefile/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,3 @@ command -v go >/dev/null 2>&1 || brew install go

# Check if the command, gum, exists, if not install it
command -v gum >/dev/null 2>&1 || go install github.com/charmbracelet/gum@latest

# Check if the command, protoc, exists, if not install it
gum spin --spinner dot --title "Installing Protoc" --show-output -- \
command -v protoc >/dev/null 2>&1 || go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28 && go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2

# Check if the command, sqlite3, exists, if not install it
gum spin --spinner dot --title "Installing SQLite3" --show-output -- \
command -v sqlite3 >/dev/null 2>&1 || brew install sqlite

# Check if the command, sqldiff, exists, if not install it
gum spin --spinner dot --title "Installing SQLDiff" --show-output -- \
command -v sqldiff >/dev/null 2>&1 || brew install sqldiff

# Check if the command, sqlc, exists, if not install it
gum spin --spinner dot --title "Installing SQLC" --show-output -- \
command -v sqlc >/dev/null 2>&1 || brew install sqlc
16 changes: 0 additions & 16 deletions scripts/taskfile/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,3 @@ command -v go >/dev/null 2>&1 || brew install go

# Check if the command, gum, exists, if not install it
command -v gum >/dev/null 2>&1 || go install github.com/charmbracelet/gum@latest

# Check if the command, protoc, exists, if not install it
gum spin --spinner dot --title "Installing Protoc" --show-output -- \
command -v protoc >/dev/null 2>&1 || go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28 && go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2

# Check if the command, sqlite3, exists, if not install it
gum spin --spinner dot --title "Installing SQLite3" --show-output -- \
command -v sqlite3 >/dev/null 2>&1 || brew install sqlite

# Check if the command, sqldiff, exists, if not install it
gum spin --spinner dot --title "Installing SQLDiff" --show-output -- \
command -v sqldiff >/dev/null 2>&1 || brew install sqldiff

# Check if the command, sqlc, exists, if not install it
gum spin --spinner dot --title "Installing SQLC" --show-output -- \
command -v sqlc >/dev/null 2>&1 || brew install sqlc
15 changes: 8 additions & 7 deletions stream.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,13 +86,14 @@ func (stream *streamReader[T]) processLines() (T, error) {
hasErrorPrefix bool
)
for {
rawLine, readErr := stream.reader.ReadBytes('\n')
if readErr != nil || hasErrorPrefix {
rawLine, err := stream.reader.ReadBytes('\n')
if err != nil || hasErrorPrefix {
respErr := stream.unmarshalError()
if respErr != nil {
return *new(T), fmt.Errorf("error, %w", respErr.Error)
return *new(T),
fmt.Errorf("error, %w", respErr.Error)
}
return *new(T), readErr
return *new(T), err
}
noSpaceLine := bytes.TrimSpace(rawLine)
if bytes.HasPrefix(noSpaceLine, errorPrefix) {
Expand All @@ -102,9 +103,9 @@ func (stream *streamReader[T]) processLines() (T, error) {
if hasErrorPrefix {
noSpaceLine = bytes.TrimPrefix(noSpaceLine, headerData)
}
writeErr := stream.errAccumulator.Write(noSpaceLine)
if writeErr != nil {
return *new(T), writeErr
err := stream.errAccumulator.Write(noSpaceLine)
if err != nil {
return *new(T), err
}
emptyMessagesCount++
if emptyMessagesCount > stream.emptyMessagesLimit {
Expand Down

0 comments on commit 0770a77

Please sign in to comment.