RePylot: Fine Tuning GPT-2

This repository contains several GPT2 fine-tuned models retrained using several programming language scripts, so as to serve as an engine for RePylot Code Generator. The code provided allows you to train GPT-2 on a custom dataset to generate text specific to your needs.

Index

• Java Code Generator
• Python Code Generator
• How to use
• Requirements

Java Code Generator

The Java Code Generator model has been fine-tuned using the source code from several popular frameworks, such as Spring or Google Gauva, and even some big projects like ElasticSearch. Apart from that, other GitHub repositories were used (See GitHub Scrapper for RePylot). We asked the model to complete public class. We obtained the following result

public class CityTests {
  @Test
  void defaultPropertyPlaceholders() {
    this.contextRunner.run

Meanwhile, the original GPT-2 model itself returned the next sequence

public class Bool is a constructor function that provides an argument that you can use to define new value types for your classes:\n\nclass Bool'}]

Other examples:

Prompt (Bigger response allowed):  public class Main implements

public class Main implements Main {

 private final String defaultName;
 private final String name;

 @Override
 public String getDefaultName() {
  return "defaultName";
 }

 public void setDefaultName(String defaultName) {
  this.defaultName = defaultName;
 }

}

Prompt:  public static void

public static void main(String[] args) throws IOException {
    final MockTerminal terminal = mockTerminal(String.class);

    final MockTerminalServices services = new MockTerminalService("netty", "netty2", "netty3");

    final SystemdPlugin.Plugin indexService = new MockSystemdPlugin(Settings.builder());
    for (SearchService searchService : services) {
         IndexSettings indexService = IndexSettingsModule.newIndexSettings(index

Some shorter answers are the following

Prompt:  @Override

  @Override
  public String toString() {
    return this.name;
  }
}
 /* * Copyright 2012-

Prompt:  import

import org.springframework.boot.actuate.endpoint.annotation.Endpoint;
import

Python Code Generator

We asked the fine-tuned model to complete from matplotlib import. We obtained the following result

from matplotlib import pyplot as plt
from sklearn.metrics import accuracy

Meanwhile, the original GPT-2 model itself returned the next sequence

from matplotlib, matplotlib.min.js <~ matplotlib.min.js >) >\n\nNote: Matplot

which is not even a Python statement.

Other examples:

Prompt:  for key

for key in key.values():
  if self.open_file(filename): # get the OpenPG

Prompt:  def inverse_sort(list, number):

def inverse_sort(list, number):
  return [list[float] for _ in range(number)]

def inverse_sort(list, number):
  return []

Prompt:  if (a ==

if (a == a[0]) 
  # check if the queue is empty 
  assert len(a)!=

How to use

You can easily run one of the given notebooks in the python directory to generate your code. There, you must look for the Model Evaluation section, and run the chunk containing the following code:

sequence = "for i, element"
generate_text(output_dir, sequence, extra_length=20)

where you can modify the sequence variable. This will print the model generation, in this case:

for i, element_name in enumerate(value_list)):
  r = random.randrange

You can also use the web version by entering into repylot website

video.mov

Requirements

Before running the scripts or notebook, please make sure you install the transformers' dependency. To do so, feel free to make use of the next command:

pip install transformers

Or simply add the next line at the beginning of your Python script

!pip install transformers

Also, you will need to have PyTorch installed, since this is what transformers' models are based in. You can install this framework using this link.