Neural Machine Translation-Seq2Seq.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"../Pics/MLSb-T.png\" width=\"160\">\n",
    "<br><br>\n",
    "<center><u><H1>Neural Machine Translation with Seq2Seq</H1></u></center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from keras.backend.tensorflow_backend import set_session\n",
    "config = tf.ConfigProto()\n",
    "config.gpu_options.allow_growth = True\n",
    "config.log_device_placement = True\n",
    "sess = tf.Session(config=config)\n",
    "set_session(sess)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary:\n",
    "1) Turn the sentences into 3 Numpy arrays, encoder_input_data, decoder_input_data, decoder_target_data:\n",
    "\n",
    "encoder_input_data is a 3D array of shape (num_pairs, max_english_sentence_length, num_english_characters) containing a one-hot vectorization of the English sentences\n",
    "decoder_input_data is a 3D array of shape (num_pairs, max_french_sentence_length, num_french_characters) containing a one-hot vectorization of the French sentences\n",
    "decoder_target_data is the same as decoder_input_data but offset by one timestep. decoder_target_data[:, t, :] will be the same as decoder_input_data[:, t + 1, :]\n",
    "\n",
    "2) Train a basic LSTM-based Seq2Seq model to predict decoder_target_data given encoder_input_data and decoder_input_data.\n",
    "\n",
    "3) Decode some sentences to check that the model is working (i.e. turn samples from encoder_input_data into corresponding samples from decoder_target_data).\n",
    "\n",
    "The original author of this code is Francois Chollet: \n",
    "\n",
    "https://github.com/keras-team/keras/blob/master/examples/lstm_seq2seq.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import Dense, LSTM, CuDNNLSTM\n",
    "from keras import Model, Input\n",
    "from keras.optimizers import Adam\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Go.\\tVe.',\n",
       " 'Go.\\tVete.',\n",
       " 'Go.\\tVaya.',\n",
       " 'Go.\\tVáyase.',\n",
       " 'Hi.\\tHola.',\n",
       " 'Run!\\t¡Corre!',\n",
       " 'Run.\\tCorred.',\n",
       " 'Who?\\t¿Quién?',\n",
       " 'Wow!\\t¡Órale!',\n",
       " 'Fire!\\t¡Fuego!',\n",
       " 'Fire!\\t¡Incendio!',\n",
       " 'Fire!\\t¡Disparad!',\n",
       " 'Help!\\t¡Ayuda!',\n",
       " 'Help!\\t¡Socorro! ¡Auxilio!',\n",
       " 'Help!\\t¡Auxilio!',\n",
       " 'Jump!\\t¡Salta!',\n",
       " 'Jump.\\tSalte.',\n",
       " 'Stop!\\t¡Parad!',\n",
       " 'Stop!\\t¡Para!',\n",
       " 'Stop!\\t¡Pare!',\n",
       " 'Wait!\\t¡Espera!',\n",
       " 'Wait.\\tEsperen.',\n",
       " 'Go on.\\tContinúa.',\n",
       " 'Go on.\\tContinúe.',\n",
       " 'Hello!\\tHola.',\n",
       " 'I ran.\\tCorrí.',\n",
       " 'I ran.\\tCorría.',\n",
       " 'I try.\\tLo intento.',\n",
       " 'I won!\\t¡He ganado!',\n",
       " 'Oh no!\\t¡Oh, no!',\n",
       " 'Relax.\\tTomátelo con soda.',\n",
       " 'Smile.\\tSonríe.',\n",
       " 'Attack!\\t¡Al ataque!',\n",
       " 'Attack!\\t¡Atacad!',\n",
       " 'Get up.\\tLevanta.',\n",
       " 'Go now.\\tVe ahora mismo.',\n",
       " 'Got it!\\t¡Lo tengo!',\n",
       " 'Got it?\\t¿Lo pillas?',\n",
       " 'Got it?\\t¿Entendiste?',\n",
       " 'He ran.\\tÉl corrió.',\n",
       " 'Hop in.\\tMétete adentro.',\n",
       " 'Hug me.\\tAbrázame.',\n",
       " 'I fell.\\tMe caí.',\n",
       " 'I know.\\tYo lo sé.',\n",
       " 'I left.\\tSalí.',\n",
       " 'I lied.\\tMentí.',\n",
       " 'I lost.\\tPerdí.',\n",
       " 'I quit.\\tDimito.',\n",
       " 'I quit.\\tRenuncié.',\n",
       " 'I work.\\tEstoy trabajando.',\n",
       " \"I'm 19.\\tTengo diecinueve.\",\n",
       " \"I'm up.\\tEstoy levantado.\",\n",
       " 'Listen.\\tEscucha.',\n",
       " 'Listen.\\tEscuche.',\n",
       " 'Listen.\\tEscuchen.',\n",
       " 'No way!\\t¡No puede ser!',\n",
       " 'No way!\\tDe ninguna manera.',\n",
       " 'No way!\\t¡De ninguna manera!',\n",
       " 'No way!\\t¡Imposible!',\n",
       " 'No way!\\t¡De ningún modo!',\n",
       " 'No way!\\t¡De eso nada!',\n",
       " 'No way!\\t¡Ni cagando!',\n",
       " 'No way!\\t¡Mangos!',\n",
       " 'No way!\\t¡Minga!',\n",
       " 'No way!\\t¡Ni en pedo!',\n",
       " 'Really?\\t¿En serio?',\n",
       " 'Really?\\t¿La verdad?',\n",
       " 'Thanks.\\t¡Gracias!',\n",
       " 'Thanks.\\tGracias.',\n",
       " 'Try it.\\tPruébalo.',\n",
       " 'We try.\\tLo procuramos.',\n",
       " 'We won.\\tGanamos.',\n",
       " 'Why me?\\t¿Por qué yo?',\n",
       " 'Ask Tom.\\tPregúntale a Tom.',\n",
       " 'Awesome!\\t¡Órale!',\n",
       " 'Be calm.\\tMantente en calma.',\n",
       " 'Be cool.\\tEstate tranquilo.',\n",
       " 'Be fair.\\tSé justo.',\n",
       " 'Be kind.\\tSean gentiles.',\n",
       " 'Be nice.\\tSé agradable.',\n",
       " 'Beat it.\\tPírate.',\n",
       " 'Call me.\\tLlamame.',\n",
       " 'Call me.\\tLlamadme.',\n",
       " 'Call me.\\tLlámame.',\n",
       " 'Call us.\\tLlámanos.',\n",
       " 'Come in.\\tEntre.',\n",
       " 'Come in.\\tPase.',\n",
       " 'Come in.\\t¡Entren!',\n",
       " 'Come on!\\t¡Órale!',\n",
       " 'Come on.\\tÁndale.',\n",
       " 'Come on.\\t¡Órale!',\n",
       " 'Drop it!\\tSuéltalo.',\n",
       " 'Get Tom.\\tAgarra a Tom.',\n",
       " 'Get out!\\tBájate.',\n",
       " 'Get out.\\tSalte.',\n",
       " 'Get out.\\tSal.',\n",
       " 'Get out.\\tSalí.',\n",
       " 'Get out.\\tSalid.',\n",
       " 'Get out.\\tSalgan.',\n",
       " 'Go away!\\t¡Vete de aquí!',\n",
       " 'Go away!\\t¡Lárgate!',\n",
       " 'Go away!\\t¡Salga de aquí!',\n",
       " 'Go away!\\t¡Largo!',\n",
       " 'Go away!\\t¡Vete ya!',\n",
       " 'Go away!\\t¡Órale!',\n",
       " 'Go away!\\t¡A la calle!',\n",
       " 'Go away.\\t¡Vete de aquí!',\n",
       " 'Go away.\\t¡Lárgate!',\n",
       " 'Go away.\\t¡Largo!',\n",
       " 'Go away.\\t¡Vete ya!',\n",
       " 'Go away.\\t¡Órale!',\n",
       " 'Go away.\\tLárguese.',\n",
       " 'Go away.\\tVáyase.',\n",
       " 'Go home.\\tVete a casa.',\n",
       " 'Go slow.\\tVaya despacio.',\n",
       " 'Goodbye!\\t¡Hasta luego!',\n",
       " 'Goodbye!\\tHasta la vista.',\n",
       " 'Goodbye!\\t¡Chau!',\n",
       " 'Hang on!\\t¡Espera!',\n",
       " 'Hang on!\\t¡Espera un momento!',\n",
       " 'Hang on!\\t¡Un segundo!',\n",
       " 'Hang on.\\tAgarra fuertemente.',\n",
       " 'He came.\\tÉl vino.',\n",
       " 'He quit.\\tÉl renunció.',\n",
       " 'Help me!\\tAyúdame.',\n",
       " 'Help me.\\tAyúdame.',\n",
       " 'Help me.\\tÉchame una mano.',\n",
       " 'Help me.\\tAyudame.',\n",
       " 'Help us.\\tAyúdanos.',\n",
       " 'Hit Tom.\\tGolpea a Tom.',\n",
       " 'Hold it!\\tEspera.',\n",
       " 'Hold on.\\tResista.',\n",
       " 'Hold on.\\tResiste.',\n",
       " 'Hold on.\\tAgarra fuertemente.',\n",
       " 'Hug Tom.\\tAbraza a Tom.',\n",
       " 'I agree.\\tEstoy de acuerdo.',\n",
       " 'I agree.\\tDe acuerdo.',\n",
       " 'I bowed.\\tMe incliné.',\n",
       " 'I moved.\\tMe he mudado.',\n",
       " 'I moved.\\tMe mudé.',\n",
       " 'I moved.\\tMe trasladé.',\n",
       " 'I moved.\\tMe he trasladado.',\n",
       " 'I slept.\\tDormí.',\n",
       " 'I tried.\\tLo intenté.',\n",
       " \"I'll go.\\tIré.\",\n",
       " \"I'm Tom.\\tSoy Tom.\",\n",
       " \"I'm fat.\\tEstoy gordo.\",\n",
       " \"I'm fat.\\tSoy gorda.\",\n",
       " \"I'm fit.\\tEstoy en forma.\",\n",
       " \"I'm hit!\\tEstoy afectado.\",\n",
       " \"I'm old.\\tSoy viejo.\",\n",
       " \"I'm shy.\\tSoy tímido.\",\n",
       " \"I'm wet.\\tEstoy mojada.\",\n",
       " \"It's OK.\\tEstá bien.\",\n",
       " \"It's me!\\tSoy yo.\",\n",
       " \"It's me.\\tSoy yo.\",\n",
       " 'Join us.\\tÚnete a nosotros.',\n",
       " 'Join us.\\tSé parte nuestra.',\n",
       " 'Keep it.\\tGuárdalo.',\n",
       " 'Me, too.\\tYo, también.',\n",
       " 'Open up.\\tAbre.',\n",
       " 'Perfect!\\t¡Perfecto!',\n",
       " 'See you.\\tNos vemos.',\n",
       " 'Show me.\\tMuéstramelo.',\n",
       " 'Show me.\\tEnséñamelo.',\n",
       " 'Show me.\\tMostrame.',\n",
       " 'Shut up!\\t¡Cierra la boca!',\n",
       " 'Shut up!\\t¡Cerrá el pico!',\n",
       " 'Skip it.\\tSáltatelo.',\n",
       " 'So long.\\tHasta la vista.',\n",
       " 'So long.\\tHasta luego.',\n",
       " 'Stop it.\\tPáralo.',\n",
       " 'Stop it.\\tDetenlo.',\n",
       " 'Take it.\\tCógelo.',\n",
       " 'Tell me.\\tDecime.',\n",
       " 'Tom ate.\\tTom comió.',\n",
       " 'Tom ran.\\tTom corrió.',\n",
       " 'Tom won.\\tTom ganó.',\n",
       " 'Wait up.\\tEspérame despierto.',\n",
       " 'Wake up!\\t¡Despierta!',\n",
       " 'Wake up!\\t¡Despiértate!',\n",
       " 'Wake up!\\t¡Despertate!',\n",
       " 'Wake up.\\tDespierta.',\n",
       " 'Wash up.\\tLávate las manos.',\n",
       " 'We care.\\tNos preocupamos.',\n",
       " 'We know.\\tLo sabemos.',\n",
       " 'We lost.\\tPerdimos.',\n",
       " 'Welcome.\\tBienvenidos.',\n",
       " 'Welcome.\\tBienvenidas.',\n",
       " 'Who ate?\\t¿Quién comió?',\n",
       " 'Who ran?\\t¿Quién corría?',\n",
       " 'Who ran?\\t¿Quién corrió?',\n",
       " 'Who won?\\t¿Quién ganó?',\n",
       " 'Who won?\\t¿Quién ha ganado?',\n",
       " 'Why not?\\t¿Por qué no?',\n",
       " 'You run.\\tCorre.',\n",
       " 'You won.\\tHas ganado.',\n",
       " 'Am I fat?\\t¿Estoy gordo?',\n",
       " 'Ask them.\\tPreguntales.',\n",
       " 'Ask them.\\tPregúntales.',\n",
       " 'Back off!\\tAparta.',\n",
       " 'Back off.\\tApártate.',\n",
       " 'Be a man.\\tSé un hombre.',\n",
       " 'Be brave.\\tSé fuerte.',\n",
       " 'Be brief.\\tSé breve.',\n",
       " 'Be brief.\\tSea breve.',\n",
       " 'Be brief.\\tSean breves.',\n",
       " 'Be quiet.\\tEstate quieto.',\n",
       " 'Be still.\\tNo te muevas.',\n",
       " 'Call Tom.\\t¡Llamalo a Tomás!',\n",
       " 'Call Tom.\\t¡Llámalo a Tomás!',\n",
       " 'Call Tom.\\t¡Llámenlo a Tomás!',\n",
       " 'Cheer up!\\tAnímate.',\n",
       " 'Cheer up.\\tVenga.',\n",
       " 'Cool off!\\t¡Cálmate!',\n",
       " 'Cuff him.\\tEspósale.',\n",
       " \"Don't go.\\tNo te vayas.\",\n",
       " 'Drive on.\\tContinúa.',\n",
       " 'Find Tom.\\tEncontralo a Tomás.',\n",
       " 'Find Tom.\\tEncuéntralo a Tomás.',\n",
       " 'Find Tom.\\tEncuéntrelo a Tomás.',\n",
       " 'Find Tom.\\tEncuéntrenlo a Tomás.',\n",
       " 'Find Tom.\\tEncuentre a Tom.',\n",
       " 'Find Tom.\\tEncuentren a Tom.',\n",
       " 'Fix this.\\tArregla esto.',\n",
       " 'Get away!\\t¡Vete de aquí!',\n",
       " 'Get away!\\t¡Lárgate!',\n",
       " 'Get away!\\t¡Largo!',\n",
       " 'Get away!\\t¡Vete ya!',\n",
       " 'Get down!\\tPonte a ello.',\n",
       " 'Get down.\\tAbajo.',\n",
       " 'Get down.\\tTúmbate.',\n",
       " 'Get lost!\\t¡Piérdete!',\n",
       " 'Get lost!\\t¡Vete de aquí!',\n",
       " 'Get lost!\\t¡Lárgate!',\n",
       " 'Get lost!\\t¡Largo!',\n",
       " 'Get lost!\\t¡Vete ya!',\n",
       " 'Get lost.\\tEsfúmate.',\n",
       " 'Get real!\\t¡Despierta!',\n",
       " 'Get real!\\t¡Piénsalo bien!',\n",
       " 'Get real.\\tAbre los ojos.',\n",
       " 'Go ahead!\\tAdelante.',\n",
       " 'Go ahead.\\tAdelante.',\n",
       " 'Go on in.\\tEntre.',\n",
       " 'Go on in.\\tEntra.',\n",
       " 'Go on in.\\tEntren.',\n",
       " 'Go on in.\\tEntrá.',\n",
       " 'Go on in.\\tEntrad.',\n",
       " 'Going up?\\t¿Subes?',\n",
       " 'Good job!\\t¡Buen trabajo!',\n",
       " 'Grab Tom.\\tSujeta a Tom.',\n",
       " 'Grab him.\\tAgárralo.',\n",
       " 'Have fun.\\tDiviértanse.',\n",
       " 'Have fun.\\tPásala bien.',\n",
       " 'Have fun.\\tPásenla bien.',\n",
       " 'He spoke.\\tÉl habló.',\n",
       " 'He tries.\\tÉl lo intenta.',\n",
       " 'He tries.\\tÉl prueba.',\n",
       " 'Help Tom.\\tAyuda a Tom.',\n",
       " 'Help him.\\tAyúdalo.',\n",
       " 'Hi, guys.\\tHola, ¿qué hay?',\n",
       " 'Hi, guys.\\tEy, chavales.',\n",
       " 'Hi, guys.\\t¿Qué pasa, troncos?',\n",
       " 'Hi, guys.\\tBuenas.',\n",
       " 'Hi, guys.\\t¿Qué tal os va?',\n",
       " 'Hi, guys.\\tHola por aquí.',\n",
       " 'How cute!\\t¡Qué encanto!',\n",
       " 'How deep?\\t¿Qué tan profundo?',\n",
       " 'How deep?\\t¿Cómo de profundo?',\n",
       " 'Humor me.\\tHazme el favor.',\n",
       " 'Humor me.\\tCompláceme.',\n",
       " 'Hurry up.\\t¡Date prisa!',\n",
       " 'Hurry up.\\tApresúrate.',\n",
       " 'I agreed.\\tConvine.',\n",
       " 'I agreed.\\tAcepté.',\n",
       " 'I agreed.\\tAccedí.',\n",
       " 'I am fat.\\tSoy gorda.',\n",
       " 'I am old.\\tEstoy viejo.',\n",
       " 'I ate it.\\tMe lo comí.',\n",
       " 'I ate it.\\tMe la comí.',\n",
       " 'I can go.\\tPuedo ir.',\n",
       " 'I did OK.\\tLo hice bien.',\n",
       " 'I did it.\\tLo hice.',\n",
       " 'I failed.\\tFracasé.',\n",
       " 'I forgot.\\tLo olvidé.',\n",
       " 'I get by.\\tMe las arreglo.',\n",
       " 'I get it.\\tLo entiendo.',\n",
       " 'I got it.\\tLo tengo.',\n",
       " 'I phoned.\\tLlamé por teléfono.',\n",
       " 'I refuse.\\tMe niego.',\n",
       " 'I resign.\\tDimito.',\n",
       " 'I resign.\\tRenuncio.',\n",
       " 'I saw it.\\tLo vi.',\n",
       " 'I smiled.\\tSonreí.',\n",
       " 'I stayed.\\tMe quedé.',\n",
       " 'I talked.\\tHablé.',\n",
       " 'I talked.\\tHablaba.',\n",
       " 'I talked.\\tCharlé.',\n",
       " 'I talked.\\tCharlaba.',\n",
       " 'I use it.\\tYo lo uso.',\n",
       " 'I waited.\\tEsperé.',\n",
       " \"I'll pay.\\tYo pagaré.\",\n",
       " \"I'm back.\\tHe vuelto.\",\n",
       " \"I'm back.\\tEstoy de vuelta.\",\n",
       " \"I'm bald.\\tSoy calvo.\",\n",
       " \"I'm bald.\\tEstoy calvo.\",\n",
       " \"I'm calm.\\tEstoy calmado.\",\n",
       " \"I'm cool.\\tEstoy tranquilo.\",\n",
       " \"I'm done.\\tHe terminado.\",\n",
       " \"I'm easy.\\tSoy bastante facilón.\",\n",
       " \"I'm fair.\\tSoy justo.\",\n",
       " \"I'm fine.\\tEstoy perfectamente.\",\n",
       " \"I'm free!\\t¡Soy libre!\",\n",
       " \"I'm free.\\tYo soy libre.\",\n",
       " \"I'm full.\\tEstoy lleno.\",\n",
       " \"I'm full.\\tEstoy llena.\",\n",
       " \"I'm full.\\tYa me llené.\",\n",
       " \"I'm here.\\tEstoy aquí.\",\n",
       " \"I'm home.\\tEstoy en casa.\",\n",
       " \"I'm hurt.\\tEstoy herido.\",\n",
       " \"I'm late.\\tLlego tarde.\",\n",
       " \"I'm lazy.\\tSoy vago.\",\n",
       " \"I'm lost.\\tEstoy perdida.\",\n",
       " \"I'm mean.\\tSoy malo.\",\n",
       " \"I'm next.\\tMe toca a mí.\",\n",
       " \"I'm okay.\\tEstoy perfectamente.\",\n",
       " \"I'm poor.\\tSoy pobre.\",\n",
       " \"I'm rich.\\tSoy rica.\",\n",
       " \"I'm rich.\\tSoy rico.\",\n",
       " \"I'm safe.\\tEstoy a salvo.\",\n",
       " \"I'm sick.\\tEstoy enferma.\",\n",
       " \"I'm thin.\\tSoy delgado.\",\n",
       " \"I'm tidy.\\tEstoy limpio.\",\n",
       " \"I'm warm.\\tEstoy calentito.\",\n",
       " \"I'm weak.\\tEstoy débil.\",\n",
       " \"I'm wise.\\tSoy listo.\",\n",
       " \"I've won.\\tGané.\",\n",
       " 'It helps.\\tEso ayuda.',\n",
       " 'It hurts.\\tDuele.',\n",
       " 'It works.\\tFunciona.',\n",
       " \"It's Tom.\\tEs Tom.\",\n",
       " \"It's fun.\\tEs divertido.\",\n",
       " \"It's his.\\tEs suyo.\",\n",
       " \"It's new.\\tEs nuevo.\",\n",
       " \"It's odd.\\tEs extraño.\",\n",
       " \"It's old.\\tEs viejo.\",\n",
       " \"It's red.\\tEs rojo.\",\n",
       " \"It's sad.\\tEs triste.\",\n",
       " 'Keep out!\\tProhibido pasar.',\n",
       " 'Keep out.\\tNo entrar.',\n",
       " 'Kiss Tom.\\tBesá a Tomás.',\n",
       " 'Kiss Tom.\\tBesa a Tomás.',\n",
       " 'Kiss Tom.\\tBesen a Tomás.',\n",
       " 'Leave it.\\tDéjalo.',\n",
       " 'Leave me.\\tDéjame.',\n",
       " 'Leave us.\\tDéjanos.',\n",
       " \"Let's go!\\tVamos.\",\n",
       " \"Let's go!\\t¡Órale!\",\n",
       " \"Let's go.\\t¡Órale!\",\n",
       " 'Look out!\\tCuidado.',\n",
       " 'Marry me.\\tCásate conmigo.',\n",
       " 'May I go?\\t¿Puedo ir?',\n",
       " 'Save Tom.\\tSalva a Tom.',\n",
       " 'She came.\\tElla vino.',\n",
       " 'She died.\\tElla murió.',\n",
       " 'She runs.\\tElla corre.',\n",
       " 'Sit down!\\t¡Sentate!',\n",
       " 'Sit down.\\tSiéntate.',\n",
       " 'Sit here.\\tSiéntate aquí.',\n",
       " 'Speak up!\\t¡Hable más fuerte!',\n",
       " 'Speak up!\\tHabla más fuerte.',\n",
       " 'Speak up!\\t¡Habla más alto!',\n",
       " 'Stand by.\\tPrepárate.',\n",
       " 'Stand by.\\tUn momento.',\n",
       " 'Stand up!\\t¡Parate!',\n",
       " 'Stand up!\\t¡De pie!',\n",
       " 'Stay put.\\tNo te muevas.',\n",
       " 'Stop Tom.\\tDetén a Tom.',\n",
       " 'Take Tom.\\tLlévate a Tom.',\n",
       " 'Tell Tom.\\tDíselo a Tom.',\n",
       " 'Terrific!\\t¡Genial!',\n",
       " 'Terrific!\\t¡Órale!',\n",
       " 'They won.\\tEllos ganaron.',\n",
       " 'Tom came.\\tTom vino.',\n",
       " 'Tom died.\\tTom ha muerto.',\n",
       " 'Tom fell.\\tTom se cayó.',\n",
       " 'Tom knew.\\tTom lo sabía.',\n",
       " 'Tom knew.\\tTom tenía constancia de ello.',\n",
       " 'Tom left.\\tTom se fue.',\n",
       " 'Tom lied.\\tTom mintió.',\n",
       " 'Tom lies.\\tTomás miente.',\n",
       " 'Tom lost.\\tTom perdió.',\n",
       " 'Tom paid.\\tTom pagó.',\n",
       " 'Tom quit.\\tTom renunció.',\n",
       " 'Tom swam.\\tTom nadó.',\n",
       " 'Tom wept.\\tTom lloró.',\n",
       " \"Tom's up.\\tTom se ha levantado.\",\n",
       " 'Too late.\\tDemasiado tarde.',\n",
       " 'Trust me.\\tConfía en mí.',\n",
       " 'Try hard.\\tInténtalo de veras.',\n",
       " 'Try some.\\tPrueba un poco.',\n",
       " 'Try some.\\tPruebe un poco.',\n",
       " 'Try some.\\tPrueben un poco.',\n",
       " 'Try this.\\tPrueben esto.',\n",
       " 'Try this.\\tProbá esto.',\n",
       " 'Try this.\\tPrueba esto.',\n",
       " 'Use this.\\tUsa esto.',\n",
       " 'Warn Tom.\\tAvísale a Tom.',\n",
       " 'Warn Tom.\\tAvísele a Tom.',\n",
       " 'Warn Tom.\\tAvísenle a Tom.',\n",
       " 'Warn Tom.\\tAvisale a Tom.',\n",
       " 'Watch me.\\tMírame.',\n",
       " 'Watch me.\\tVigílame.',\n",
       " 'Watch me.\\tObsérvame.',\n",
       " 'Watch us.\\tObsérvalos.',\n",
       " 'Watch us.\\tObsérvanos.',\n",
       " 'We agree.\\tEstamos de acuerdo.',\n",
       " 'We tried.\\tLo intentamos.',\n",
       " \"We'll go.\\tIremos.\",\n",
       " \"We're OK.\\tEstamos bien.\",\n",
       " 'What for?\\t¿Para qué?',\n",
       " 'What fun!\\t¡Qué divertido!',\n",
       " 'Who am I?\\t¿Quién soy yo?',\n",
       " 'Who came?\\t¿Quién vino?',\n",
       " 'Who died?\\t¿Quién murió?',\n",
       " 'Who fell?\\t¿Quién se calló?',\n",
       " 'Who fell?\\t¿Quién se cayó?',\n",
       " 'Who quit?\\t¿Quién se ha ido?',\n",
       " 'Who quit?\\t¿Quién lo ha dejado?',\n",
       " 'Who swam?\\t¿Quién nadó?',\n",
       " \"Who's he?\\t¿Quién es él?\",\n",
       " 'Write me.\\tEscríbeme.',\n",
       " 'After you.\\tDespués de ti.',\n",
       " 'After you.\\tTú primero.',\n",
       " 'After you.\\tUsted primero.',\n",
       " 'After you.\\tDespués de usted.',\n",
       " 'After you.\\tDespués de vosotras.',\n",
       " 'Aim. Fire!\\tApunta. ¡Fuego!',\n",
       " 'Answer me.\\tRespóndeme.',\n",
       " 'Answer me.\\tRespondedme.',\n",
       " 'Answer me.\\tRespóndanme.',\n",
       " 'Birds fly.\\tLos pájaros vuelan.',\n",
       " 'Bless you.\\tJesús.',\n",
       " 'Call home!\\t¡Llama a casa!',\n",
       " 'Calm down.\\tCálmate.',\n",
       " 'Calm down.\\tTranquilízate.',\n",
       " 'Calm down.\\tTomátelo con soda.',\n",
       " 'Calm down.\\tBajá un cambio.',\n",
       " 'Can we go?\\t¿Podemos ir?',\n",
       " 'Can we go?\\t¿Podemos irnos?',\n",
       " 'Can we go?\\t¿Podemos marcharnos?',\n",
       " 'Catch Tom.\\t¡Atrápelo a Tomás!',\n",
       " 'Catch him.\\tCaptúralo.',\n",
       " 'Catch him.\\tCaptúrenlo.',\n",
       " 'Catch him.\\tAtrápenlo.',\n",
       " 'Catch him.\\tAtrápalo.',\n",
       " 'Come back.\\tVuelve.',\n",
       " 'Come here.\\tVení.',\n",
       " 'Come here.\\tVen.',\n",
       " 'Come here.\\tVenid aquí.',\n",
       " 'Come home.\\tVen a casa.',\n",
       " 'Come over.\\tVenid aquí.',\n",
       " 'Come over.\\tVente.',\n",
       " 'Come over.\\tVeníos.',\n",
       " 'Come over.\\tVenga aquí.',\n",
       " 'Come soon.\\tVuelve pronto.',\n",
       " 'Do it now.\\tHazlo ahora.',\n",
       " 'Dogs bark.\\tLos perros ladran.',\n",
       " \"Don't ask.\\tNo preguntes.\",\n",
       " \"Don't cry.\\tNo llores.\",\n",
       " \"Don't cry.\\t¡No llorés!\",\n",
       " \"Don't cry.\\t¡No llores!\",\n",
       " \"Don't cry.\\tNo lloren.\",\n",
       " \"Don't die.\\t¡No te mueras!\",\n",
       " \"Don't lie.\\tNo mintáis.\",\n",
       " \"Don't run.\\tNo corras.\",\n",
       " 'Fantastic!\\t¡Fantástico!',\n",
       " 'Fantastic!\\t¡Es estupendo!',\n",
       " 'Fantastic!\\t¡Órale!',\n",
       " 'Feel this.\\tTienta esto.',\n",
       " 'Feel this.\\tTiente esto.',\n",
       " 'Feel this.\\tTienten esto.',\n",
       " 'Feel this.\\tTentad esto.',\n",
       " 'Feel this.\\tTentá esto.',\n",
       " 'Follow me.\\tSígueme.',\n",
       " 'Follow me.\\tSeguidme.',\n",
       " 'Follow us.\\tSíguenos.',\n",
       " 'Forget it.\\tDéjalo.',\n",
       " 'Forget me.\\tOlvídame.',\n",
       " 'Forget me.\\tOlvídate de mí.',\n",
       " 'Forget me.\\tOlvídenme.',\n",
       " 'Forget me.\\tOlvídense de mí.',\n",
       " 'Get ready.\\tPreparaos.',\n",
       " 'Go for it.\\tVe a por ello.',\n",
       " 'Go for it.\\tVe a cogerlo.',\n",
       " 'Go get it.\\tVe a por ello.',\n",
       " 'Go get it.\\tVete a por ello.',\n",
       " 'Go get it.\\tId a por ello.',\n",
       " 'Go inside.\\tEntra.',\n",
       " 'Go to bed.\\tVete a la cama.',\n",
       " 'Grab that.\\tCógelo.',\n",
       " 'Grab this.\\tAgarra esto.',\n",
       " 'Have some.\\tCoge alguno.',\n",
       " 'Have some.\\tToma algo.',\n",
       " 'Have some.\\tTome alguno.',\n",
       " 'He is old.\\tÉl es viejo.',\n",
       " 'He is old.\\tÉl es anciano.',\n",
       " 'He shaved.\\tÉl se afeitó.',\n",
       " 'He smiled.\\tSonrió.',\n",
       " \"He's a DJ.\\tÉl es DJ.\",\n",
       " \"He's fast.\\tÉl es veloz.\",\n",
       " \"He's good.\\tÉl es bueno.\",\n",
       " \"He's rich.\\tÉl es rico.\",\n",
       " \"He's rich.\\tEs rico.\",\n",
       " 'Here I am.\\tAcá estoy.',\n",
       " 'Hold this.\\tSostén esto.',\n",
       " 'How awful!\\t¡Qué horror!',\n",
       " 'How weird!\\t¡Qué raro!',\n",
       " \"How's Tom?\\t¿Cómo está Tom?\",\n",
       " 'Humor Tom.\\tDivierte a Tom.',\n",
       " 'I am busy.\\tEstoy ocupada.',\n",
       " 'I am full.\\tEstoy lleno.',\n",
       " 'I am full.\\tEstoy llena.',\n",
       " 'I am good.\\tSoy bueno.',\n",
       " 'I am here.\\tEstoy aquí.',\n",
       " 'I am sick.\\tEstoy enferma.',\n",
       " 'I am weak.\\tEstoy débil.',\n",
       " 'I beg you.\\tTe lo ruego.',\n",
       " 'I beg you.\\tTe lo suplico.',\n",
       " 'I can fly.\\tPuedo volar.',\n",
       " 'I can run.\\tPuedo correr.',\n",
       " 'I can run.\\tSé correr.',\n",
       " 'I can ski.\\tSé esquiar.',\n",
       " 'I cringed.\\tSentí vergüenza.',\n",
       " 'I fainted.\\tPerdí la consciencia.',\n",
       " 'I fainted.\\tQuedé inconsciente.',\n",
       " 'I gave up.\\tLo dejé.',\n",
       " 'I gave up.\\tMe rendí.',\n",
       " 'I gave up.\\tTiré la toalla.',\n",
       " 'I gave up.\\tCedí.',\n",
       " 'I get you.\\tTe entiendo.',\n",
       " 'I get you.\\tTe pillo.',\n",
       " 'I get you.\\tYa te pillo.',\n",
       " 'I got hit.\\tMe dieron.',\n",
       " 'I got hit.\\tMe alcanzaron.',\n",
       " 'I hate it.\\tLo odio.',\n",
       " 'I hate it.\\tMe la seca.',\n",
       " 'I hate it.\\tMe la baja.',\n",
       " 'I hit Tom.\\tGolpeo a Tom.',\n",
       " 'I hope so.\\tEso espero.',\n",
       " 'I knew it.\\tLo sabía.',\n",
       " 'I laughed.\\tMe reí.',\n",
       " 'I like it.\\tMe gusta.',\n",
       " 'I love it!\\t¡Lo adoro!',\n",
       " 'I love it.\\tMe encanta.',\n",
       " 'I mean it!\\tHablo en serio.',\n",
       " 'I mean it.\\tLo digo en serio.',\n",
       " 'I mean it.\\tHablo en serio.',\n",
       " 'I miss it.\\tLo echo en falta.',\n",
       " 'I miss it.\\tLo echo de menos.',\n",
       " 'I miss it.\\tLo añoro.',\n",
       " 'I need it.\\tLo necesito.',\n",
       " 'I saw Tom.\\tVi a Tom.',\n",
       " 'I saw him.\\tLe vi.',\n",
       " 'I saw him.\\tLo vi.',\n",
       " 'I saw one.\\tVi uno.',\n",
       " 'I saw one.\\tVi a uno.',\n",
       " 'I saw one.\\tVeía a uno.',\n",
       " 'I saw you.\\tTe vi.',\n",
       " 'I saw you.\\tTe he visto.',\n",
       " 'I see Tom.\\tVeo a Tom.',\n",
       " 'I tripped.\\tMe tropecé.',\n",
       " \"I'll cook.\\tYo cocinaré.\",\n",
       " \"I'll cook.\\tYo guisaré.\",\n",
       " \"I'll live.\\tViviré.\",\n",
       " \"I'll sing.\\tCantaré.\",\n",
       " \"I'll stop.\\tPararé.\",\n",
       " \"I'll stop.\\tLo dejaré.\",\n",
       " \"I'll wait.\\tEsperaré.\",\n",
       " \"I'll wait.\\tAguardaré.\",\n",
       " \"I'll walk.\\tCaminaré.\",\n",
       " \"I'll walk.\\tAndaré.\",\n",
       " \"I'll work.\\tVoy a trabajar.\",\n",
       " \"I'm a man.\\tSoy un hombre.\",\n",
       " \"I'm a pro.\\tSoy un pro.\",\n",
       " \"I'm alone.\\tEstoy solo.\",\n",
       " \"I'm alone.\\tEstoy sola.\",\n",
       " \"I'm alone.\\tYo estoy solo.\",\n",
       " \"I'm alone.\\tYo estoy sola.\",\n",
       " \"I'm angry.\\tEstoy enojado.\",\n",
       " \"I'm angry.\\tEstoy enojada.\",\n",
       " \"I'm awake.\\tEstoy despierto.\",\n",
       " \"I'm blind.\\tSoy ciega.\",\n",
       " \"I'm broke.\\tEstoy sin dinero.\",\n",
       " \"I'm broke.\\tEstoy quebrado.\",\n",
       " \"I'm broke.\\tEstoy sin blanca.\",\n",
       " \"I'm broke.\\tNo tengo blanca.\",\n",
       " \"I'm crazy.\\tEstoy loco.\",\n",
       " \"I'm drunk.\\tEstoy borracho.\",\n",
       " \"I'm drunk.\\tEstoy tomada.\",\n",
       " \"I'm drunk.\\tEstoy prendida.\",\n",
       " \"I'm dying.\\tMe estoy muriendo.\",\n",
       " \"I'm first.\\tSoy el primero.\",\n",
       " \"I'm first.\\tVoy el primero.\",\n",
       " \"I'm first.\\tMe toca primero.\",\n",
       " \"I'm first.\\tYo voy primero.\",\n",
       " \"I'm happy.\\tSoy feliz.\",\n",
       " \"I'm happy.\\tEstoy feliz.\",\n",
       " \"I'm loved.\\tSoy amado.\",\n",
       " \"I'm loved.\\tSoy amada.\",\n",
       " \"I'm obese.\\tSoy obeso.\",\n",
       " \"I'm ready.\\tEstoy listo.\",\n",
       " \"I'm sorry.\\tLo siento.\",\n",
       " \"I'm tired.\\tYo estoy cansado.\",\n",
       " \"I'm tired.\\tYo estoy cansada.\",\n",
       " \"I'm yours.\\tSoy tuyo.\",\n",
       " \"I'm yours.\\tSoy tuya.\",\n",
       " \"I'm yours.\\tSoy suyo.\",\n",
       " \"I've lost.\\tHe perdido.\",\n",
       " 'Ignore it.\\tÓlvidalo.',\n",
       " 'Ignore it.\\tNo hagas caso.',\n",
       " 'Is Tom OK?\\t¿Está bien Tom?',\n",
       " 'Is Tom in?\\t¿Está Tom?',\n",
       " 'Is it bad?\\t¿Es grave?',\n",
       " 'Is it bad?\\t¿Es malo?',\n",
       " 'Is it far?\\t¿Está lejos?',\n",
       " 'Is it you?\\t¿Eres tú?',\n",
       " 'It burned.\\tSe quemó.',\n",
       " 'It failed.\\tNo lo consiguió.',\n",
       " 'It failed.\\tFalló.',\n",
       " 'It failed.\\tFracasó.',\n",
       " 'It is new.\\tEs nuevo.',\n",
       " 'It rained.\\tLlovió.',\n",
       " 'It snowed.\\tNevó.',\n",
       " 'It stinks.\\tApesta.',\n",
       " 'It worked.\\tFuncionó.',\n",
       " \"It's 3:10.\\tSon las tres y diez.\",\n",
       " \"It's 8:30.\\tSon las 8:30.\",\n",
       " \"It's a TV.\\tEs un televisor.\",\n",
       " \"It's cold.\\tHace frío.\",\n",
       " \"It's cold.\\tEstá frío.\",\n",
       " \"It's cool.\\tEstá frío.\",\n",
       " \"It's cool.\\tEstá fresco.\",\n",
       " \"It's dark.\\tEstá oscuro.\",\n",
       " \"It's done!\\t¡Está listo!\",\n",
       " \"It's fine.\\tEstá perfecto.\",\n",
       " \"It's good.\\tEstá bien.\",\n",
       " \"It's good.\\tEs bueno.\",\n",
       " \"It's here.\\tEstá aquí.\",\n",
       " \"It's hers.\\tEs de ella.\",\n",
       " \"It's late.\\tEs tarde.\",\n",
       " \"It's mine.\\tEs mío.\",\n",
       " \"It's mine.\\tEs mía.\",\n",
       " \"It's nice.\\tEs agradable.\",\n",
       " \"It's nice.\\tHace buen tiempo.\",\n",
       " \"It's okay.\\tVale.\",\n",
       " \"It's okay.\\tEstá bien.\",\n",
       " \"It's ours.\\tEs nuestro.\",\n",
       " \"It's ours.\\tEs la nuestra.\",\n",
       " \"It's over.\\tSe terminó.\",\n",
       " \"It's over.\\tSe acabó.\",\n",
       " \"It's time.\\tHa llegado el momento.\",\n",
       " \"It's time.\\tEs la hora.\",\n",
       " \"It's time.\\tEs el momento.\",\n",
       " \"It's time.\\tHa llegado la hora.\",\n",
       " \"It's true!\\t¡Es verdad!\",\n",
       " \"It's true.\\tEs verdad.\",\n",
       " \"It's work.\\tEs mi trabajo.\",\n",
       " 'Keep them.\\tGuárdalos.',\n",
       " 'Keep them.\\tQuédatelos.',\n",
       " 'Keep them.\\tGuárdenselos.',\n",
       " 'Keep this.\\tGuarda esto.',\n",
       " 'Keep this.\\tGuarde esto.',\n",
       " 'Keep this.\\tGuardá esto.',\n",
       " 'Keep this.\\tGuardad esto.',\n",
       " 'Keep this.\\tGuarden esto.',\n",
       " 'Keep warm.\\tMantente caliente.',\n",
       " 'Keep warm.\\tManténgase caliente.',\n",
       " 'Keep warm.\\tMantenete caliente.',\n",
       " 'Keep warm.\\tMantenete abrigado.',\n",
       " 'Keep warm.\\tMantente abrigado.',\n",
       " 'Keep warm.\\tManténgase abrigado.',\n",
       " 'Keep warm.\\tManténganse calientes.',\n",
       " 'Keep warm.\\tManténganse abrigados.',\n",
       " 'Leave Tom.\\tDejalo a Tomás.',\n",
       " 'Leave Tom.\\tDéjalo a Tomás.',\n",
       " 'Leave Tom.\\tDéjelo a Tomás.',\n",
       " 'Leave now.\\tAhora márchate.',\n",
       " 'Leave now.\\tAhora vete.',\n",
       " 'Leave now.\\tAhora lárgate.',\n",
       " 'Let me go!\\t¡Déjame marchar!',\n",
       " 'Let me go!\\t¡Suéltame!',\n",
       " 'Let me go!\\t¡Déjeme ir!',\n",
       " 'Let me go.\\t¡Déjeme ir!',\n",
       " 'Let me in.\\tDéjame entrar.',\n",
       " 'Let me in.\\tDéjeme entrar.',\n",
       " 'Let us in.\\tDéjanos entrar.',\n",
       " \"Let's eat.\\tComamos.\",\n",
       " \"Let's see.\\tVeamos.\",\n",
       " \"Let's try!\\t¡Intentémoslo!\",\n",
       " 'Lie still.\\tTúmbate en silencio.',\n",
       " 'Lie still.\\tMantente quieto en el suelo.',\n",
       " 'Listen up.\\tEscuchá.',\n",
       " 'Look away.\\tMirá para otro lado.',\n",
       " 'Look away.\\tMira para otro lado.',\n",
       " 'Look away.\\tMire para otro lado.',\n",
       " 'Look away.\\tMiren para otro lado.',\n",
       " 'Look back!\\t¡Mira hacia atrás!',\n",
       " 'Look back.\\tMira atrás.',\n",
       " 'Look back.\\tMire atrás.',\n",
       " 'Look back.\\tMirad atrás.',\n",
       " 'Look here.\\tMira aquí.',\n",
       " 'Look here.\\tMirad aquí.',\n",
       " 'Look here.\\tMiren aquí.',\n",
       " 'Loosen it.\\tAflójalo.',\n",
       " 'Loosen it.\\tSuéltalo.',\n",
       " 'Loosen it.\\tSuéltala.',\n",
       " 'Loosen it.\\tAflójala.',\n",
       " 'Move over.\\tHazte a un lado.',\n",
       " 'Move over.\\tDeja sitio.',\n",
       " 'Move over.\\tMueve el culo.',\n",
       " 'Move over.\\tDeja pasar.',\n",
       " 'Nice shot!\\t¡Buen tiro!',\n",
       " 'Of course!\\t¡Por supuesto!',\n",
       " 'Open fire!\\t¡Disparad!',\n",
       " 'Open fire!\\t¡Fuego a discreción!',\n",
       " 'Pardon me?\\t¿Perdona?',\n",
       " 'Please go.\\tPor favor, ve.',\n",
       " 'Please go.\\tPor favor, vete.',\n",
       " 'Please go.\\tPor favor, márchate.',\n",
       " 'Please go.\\tPor favor, váyase.',\n",
       " 'Put it on.\\tPóntelo.',\n",
       " 'Put it on.\\tPóngaselo.',\n",
       " 'Read this.\\tLee esto.',\n",
       " 'Read this.\\tLea esto.',\n",
       " 'Read this.\\tLean esto.',\n",
       " 'Read this.\\tLeed esto.',\n",
       " 'Say hello.\\tDecí hola.',\n",
       " 'Search me.\\tA mí que me registren.',\n",
       " 'See above.\\tVéase arriba.',\n",
       " 'See above.\\tVéase más arriba.',\n",
       " 'Seize him!\\t¡A por él!',\n",
       " 'Seize him!\\t¡Cogedlo!',\n",
       " 'Seize him!\\t¡Arrestadlo!',\n",
       " 'Seize him!\\t¡Prendedlo!',\n",
       " 'Seize him!\\t¡Capturadlo!',\n",
       " 'Seriously?\\t¿En serio?',\n",
       " 'She tried.\\tElla lo intentó.',\n",
       " 'She tried.\\tElla lo probó.',\n",
       " 'She walks.\\tElla anda.',\n",
       " 'She walks.\\tElla camina.',\n",
       " 'She walks.\\tAnda.',\n",
       " \"She's hot.\\tElla es candente.\",\n",
       " \"She's hot.\\tEstá como un tren.\",\n",
       " \"She's hot.\\tEstá de toma pan y moja.\",\n",
       " \"She's hot.\\tEstá buena.\",\n",
       " \"She's hot.\\tEstá que cruje.\",\n",
       " 'Sign here.\\tFirme aquí.',\n",
       " 'Sign this.\\tFirma esto.',\n",
       " 'Sign this.\\tFirme esto.',\n",
       " 'Sign this.\\tFirmen esto.',\n",
       " 'Sign this.\\tFirmá esto.',\n",
       " 'Sign this.\\tFirmad esto.',\n",
       " 'Sit still.\\tSiéntate y no te muevas.',\n",
       " 'Sit still.\\tSiéntate y quédate quieto.',\n",
       " 'Sit there.\\tSentate ahí.',\n",
       " 'Sit tight.\\tTú mantente.',\n",
       " 'Slow down.\\tQuieto.',\n",
       " 'Slow down.\\tMás despacio.',\n",
       " 'Stay away.\\tFuera.',\n",
       " 'Stay away.\\tAléjate.',\n",
       " 'Stay calm.\\tMantén la calma.',\n",
       " 'Stay calm.\\tMantené la calma.',\n",
       " 'Stay here.\\tQuedate acá.',\n",
       " 'Stay thin.\\tMantente delgado.',\n",
       " 'Step back.\\tDa un paso atrás.',\n",
       " 'Stop here.\\tPara aquí.',\n",
       " 'Stop here.\\tPárate aquí.',\n",
       " 'Stop here.\\tDetente aquí.',\n",
       " 'Stop here.\\tDeténgase aquí.',\n",
       " 'Stop here.\\tDeténganse aquí.',\n",
       " 'Stop here.\\tParaos aquí.',\n",
       " 'Stop here.\\tParad aquí.',\n",
       " 'Stop that.\\tPara eso.',\n",
       " 'Stop them.\\tDetenlos.',\n",
       " 'Stop them.\\tPárales.',\n",
       " 'Take care!\\t¡Cuidate!',\n",
       " 'Take care!\\t¡Te cuidas!',\n",
       " 'Take care.\\tCuídate.',\n",
       " 'Take care.\\tCuidaos.',\n",
       " 'Take mine.\\tCoge el mío.',\n",
       " 'Take mine.\\tToma el mío.',\n",
       " 'Take over.\\tAsume el mando.',\n",
       " 'Take over.\\tToma el control.',\n",
       " 'Take over.\\tEncárgate tú.',\n",
       " 'Take this.\\tToma esto.',\n",
       " 'Take this.\\tTome esto.',\n",
       " 'Take this.\\tTomen esto.',\n",
       " 'Take this.\\tTomá esto.',\n",
       " 'Thank you.\\t¡Gracias!',\n",
       " 'Thank you.\\tGracias a ti.',\n",
       " 'Thank you.\\tGracias.',\n",
       " \"That's it.\\tEso es.\",\n",
       " \"That's me.\\tSoy yo.\",\n",
       " \"That's me.\\tEse soy yo.\",\n",
       " 'Then what?\\t¿Entonces qué?',\n",
       " 'They fell.\\tCayeron.',\n",
       " 'They left.\\tSe fueron.',\n",
       " 'They left.\\tSe marcharon.',\n",
       " 'They left.\\tSe iban.',\n",
       " 'They lost.\\tPerdieron.',\n",
       " 'Tom bowed.\\tTom se inclinó.',\n",
       " 'Tom cared.\\tTom se preocupó.',\n",
       " 'Tom cared.\\tTom se preocupaba.',\n",
       " 'Tom cared.\\tA Tom le importaba.',\n",
       " 'Tom cares.\\tTom se preocupa.',\n",
       " 'Tom cares.\\tA Tom le importa.',\n",
       " 'Tom cried.\\tTom lloró.',\n",
       " 'Tom dozed.\\tTom dormitó.',\n",
       " 'Tom drove.\\tTomás condujo.',\n",
       " 'Tom is OK.\\tTom está bien.',\n",
       " 'Tom is in.\\tTom está dentro.',\n",
       " 'Tom is up.\\tTom está levantado.',\n",
       " 'Tom knits.\\tTom teje.',\n",
       " 'Tom knows.\\tTom sabe.',\n",
       " 'Tom moved.\\tTom se mudó.',\n",
       " 'Tom moved.\\tTom se movió.',\n",
       " 'Tom rocks.\\tTom se mece.',\n",
       " 'Tom stood.\\tTom se quedó.',\n",
       " 'Tom swims.\\tTom nada.',\n",
       " 'Tom swore.\\tTom juró.',\n",
       " 'Tom tried.\\tTomás lo intentó.',\n",
       " 'Tom tries.\\tTomás lo intenta.',\n",
       " 'Tom voted.\\tTomás votó.',\n",
       " 'Tom walks.\\tTom camina.',\n",
       " 'Tom waved.\\tTom saludó.',\n",
       " 'Tom works.\\tTomás trabaja.',\n",
       " \"Tom'll go.\\tTom irá.\",\n",
       " \"Tom's fat.\\tTom está gordo.\",\n",
       " \"Tom's mad.\\tTom está loco.\",\n",
       " \"Tom's sad.\\tTom está triste.\",\n",
       " \"Tom's shy.\\tTom es tímido.\",\n",
       " 'Trust Tom.\\tConfía en Tom.',\n",
       " 'Try again.\\tInténtalo de nuevo.',\n",
       " 'Try again.\\tIntentadlo otra vez.',\n",
       " 'Try again.\\tIntenta otra vez.',\n",
       " 'Try it on.\\tPruébatelo.',\n",
       " 'Wait here.\\tEspera aquí.',\n",
       " 'Wait here.\\tAguarda aquí.',\n",
       " 'Wait here.\\tEsperen aquí.',\n",
       " 'Wait here.\\tEsperad aquí.',\n",
       " 'Watch Tom.\\tVigila a Tom.',\n",
       " 'Watch out!\\t¡Abusado!',\n",
       " 'We can go.\\tPodemos ir.',\n",
       " 'We can go.\\tPodemos irnos.',\n",
       " 'We can go.\\tNos podemos marchar.',\n",
       " 'We failed.\\tFallamos.',\n",
       " 'We failed.\\tHemos fallado.',\n",
       " 'We forgot.\\tNos olvidamos.',\n",
       " 'We saw it.\\tLo vimos.',\n",
       " 'We saw it.\\tLo veíamos.',\n",
       " 'We talked.\\tConversamos.',\n",
       " 'We waited.\\tNosotros esperamos.',\n",
       " \"We'll see.\\tDespués veremos.\",\n",
       " \"We'll see.\\tYa veremos.\",\n",
       " \"We'll try.\\tLo intentaremos.\",\n",
       " \"We've won!\\t¡Hemos ganado!\",\n",
       " \"What's up?\\t¿Qué hay?\",\n",
       " 'Who cares?\\t¿A quién le importa?',\n",
       " 'Who is he?\\t¿Quién es él?',\n",
       " 'Who is it?\\t¿Quién es?',\n",
       " 'Who knows?\\tAabe...',\n",
       " 'Who stood?\\t¿Quién se quedó?',\n",
       " \"Who'll go?\\t¿Quién irá?\",\n",
       " \"Who's Tom?\\t¿Quién es Tom?\",\n",
       " 'Write Tom.\\tEscribe a Tom.',\n",
       " 'You drive.\\tTú maneja.',\n",
       " 'You idiot!\\t¡Pedazo de idiota!',\n",
       " 'You start.\\tTú empieza.',\n",
       " 'You tried.\\tLo intentaste.',\n",
       " \"You're OK.\\tTú estás perfectamente.\",\n",
       " \"You're OK.\\tEstás perfectamente.\",\n",
       " 'Aim higher.\\tMás alto.',\n",
       " 'Aim higher.\\tApuntad más alto.',\n",
       " 'All aboard!\\t¡Todos a bordo!',\n",
       " 'Am I right?\\t¿Tengo razón?',\n",
       " 'Am I wrong?\\t¿Me equivoco?',\n",
       " 'Am I wrong?\\t¿Estoy equivocada?',\n",
       " 'Answer Tom.\\t¡Respóndale a Tomás!',\n",
       " 'Answer Tom.\\t¡Respondele a Tomás!',\n",
       " 'Answer Tom.\\t¡Respóndanle a Tomás!',\n",
       " 'Are you 18?\\t¿Tienes dieciocho?',\n",
       " 'Are you 18?\\t¿Has cumplido dieciocho?',\n",
       " 'Are you OK?\\t¿Estás bien?',\n",
       " 'Are you in?\\t¿Te apuntas?',\n",
       " 'Are you in?\\t¿Estás dentro?',\n",
       " 'Are you up?\\t¿Estás levantado?',\n",
       " 'Ask anyone.\\tPregúntale a cualquiera.',\n",
       " 'Ask anyone.\\tPreguntadle a cualquiera.',\n",
       " 'Ask anyone.\\tPregúntele a cualquiera.',\n",
       " 'Ask anyone.\\tPregúntenle a cualquiera.',\n",
       " 'Ask around.\\tPregunta por aquí.',\n",
       " 'Ask around.\\tPregunta en los alrededores.',\n",
       " 'Be careful.\\t¡Ten cuidado!',\n",
       " 'Be careful.\\t¡Sé cuidadoso!',\n",
       " 'Be content.\\tEstate contento.',\n",
       " 'Be on time.\\tLlega a tiempo.',\n",
       " 'Be on time.\\tLlegue a tiempo.',\n",
       " 'Be patient.\\tSea paciente.',\n",
       " 'Be serious.\\tSé serio.',\n",
       " 'Birds sing.\\tLos pájaros cantan.',\n",
       " 'Birds sing.\\tLos pájaros están cantando.',\n",
       " 'Bring food.\\tTraed comida.',\n",
       " 'Bring help.\\tTraed ayuda.',\n",
       " 'Bring wine.\\tTrae vino.',\n",
       " 'Can I come?\\t¿Puedo ir?',\n",
       " 'Can I come?\\t¿Puedo venir?',\n",
       " 'Can I come?\\t¿Puedo acercarme?',\n",
       " 'Can I help?\\t¿Puedo ayudar?',\n",
       " 'Can I stay?\\t¿Me puedo quedar?',\n",
       " 'Carry this.\\tLleva esto.',\n",
       " 'Check that.\\tComprobad eso.',\n",
       " 'Check this.\\tComprueba esto.',\n",
       " 'Choose one.\\tEscoge uno.',\n",
       " 'Come again.\\tVuelve otra vez.',\n",
       " 'Come alone.\\tVen solo.',\n",
       " 'Come along.\\tVente.',\n",
       " 'Come along.\\tVénganse.',\n",
       " 'Come early.\\tVení temprano.',\n",
       " 'Come early.\\tVen temprano.',\n",
       " 'Come early.\\tVengan temprano.',\n",
       " 'Come early.\\tVenga temprano.',\n",
       " 'Come on in!\\t¡Pásale!',\n",
       " 'Come on in!\\t¡Pásele!',\n",
       " 'Come on in!\\t¡Pásenle!',\n",
       " 'Come on in.\\tEntre.',\n",
       " 'Come on in.\\tPase.',\n",
       " 'Come quick!\\t¡Ven rápido!',\n",
       " 'Come quick!\\t¡Vení rápido!',\n",
       " 'Come to me.\\tVen a mí.',\n",
       " 'Come to me.\\tVenid a mí.',\n",
       " 'Come to us.\\tVen a nosotros.',\n",
       " 'Come to us.\\tVenid a nosotros.',\n",
       " 'Cut it out!\\t¡Ya párale!',\n",
       " 'Did Tom go?\\t¿Fue Tom?',\n",
       " 'Do come in!\\t¡Pásale!',\n",
       " 'Do come in!\\t¡Pásele!',\n",
       " 'Do come in!\\t¡Pásenle!',\n",
       " 'Do come in.\\tPasa adentro.',\n",
       " 'Do come in.\\tEntra de una vez.',\n",
       " 'Do come in.\\tEntra ya.',\n",
       " 'Do come in.\\tMétete dentro.',\n",
       " 'Do men cry?\\t¿Los hombres lloran?',\n",
       " \"Don't come.\\tNo vengas.\",\n",
       " \"Don't jump!\\t¡No saltéis!\",\n",
       " \"Don't look.\\tNo miréis.\",\n",
       " \"Don't move!\\t¡No os mováis!\",\n",
       " \"Don't move.\\tNo te muevas.\",\n",
       " \"Don't move.\\tNo se mueva.\",\n",
       " \"Don't move.\\tNo se muevan.\",\n",
       " \"Don't sing.\\tNo cantes.\",\n",
       " \"Don't sing.\\tNo canten.\",\n",
       " \"Don't stop.\\tNo pares.\",\n",
       " \"Don't talk!\\t¡No hables!\",\n",
       " \"Don't talk.\\tNo hables.\",\n",
       " \"Don't wait.\\t¡No esperes!\",\n",
       " \"Don't wait.\\t¡No esperen!\",\n",
       " \"Don't wait.\\tNo esperes.\",\n",
       " \"Don't yell.\\tNo grites.\",\n",
       " 'Eat slowly.\\tCome despacio.',\n",
       " 'Eat slowly.\\t¡Comé despacio!',\n",
       " 'Fire burns.\\tEl fuego quema.',\n",
       " 'Follow Tom.\\tSeguilo a Tomás.',\n",
       " 'Follow Tom.\\tSíguelo a Tomás.',\n",
       " 'Follow Tom.\\tSígalo a Tomás.',\n",
       " 'Follow Tom.\\tSíganlo a Tomás.',\n",
       " 'Follow him.\\tSíguele.',\n",
       " 'Follow him.\\tSíguelo.',\n",
       " 'Forget Tom.\\tOlvidate de Tomás.',\n",
       " 'Forget Tom.\\tOlvídate de Tomás.',\n",
       " 'Forget Tom.\\tOlvídese de Tomás.',\n",
       " 'Forget him.\\tOlvídenlo.',\n",
       " 'Forgive us.\\tPerdónanos.',\n",
       " 'Forgive us.\\tPerdónenos.',\n",
       " 'Get a life.\\tConsíguete una vida.',\n",
       " 'Get inside.\\tEntra.',\n",
       " 'Get to bed.\\tVete a la cama.',\n",
       " 'Give it up.\\tDéjalo.',\n",
       " 'Go on home.\\tVete a casa.',\n",
       " 'Go on home.\\tVáyase a casa.',\n",
       " 'Go see Tom.\\tVe a ver a Tom.',\n",
       " 'Go to work.\\tAndá a trabajar.',\n",
       " 'God exists.\\tDios existe.',\n",
       " 'Have faith.\\tTen fe.',\n",
       " 'Have faith.\\tTened fe.',\n",
       " 'Have faith.\\tTengan fe.',\n",
       " 'Have faith.\\tConfíen.',\n",
       " 'He ate out.\\tÉl salió a comer.',\n",
       " 'He coughed.\\tTosió.',\n",
       " 'He gave in.\\tÉl se rindió.',\n",
       " ...]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lines = open('../data/machine_translation/spa.txt', encoding='utf-8').read().split('\\n')\n",
    "lines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "119937"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(lines)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating English - Spanish Sentences:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "eng_sent = []\n",
    "spa_sent = []\n",
    "eng_chars = set()\n",
    "spa_chars = set()\n",
    "n_samples = 10000\n",
    "\n",
    "for line in range(n_samples):\n",
    "    \n",
    "    eng_line = str(lines[line]).split('\\t')[0]\n",
    "    \n",
    "    # Append '\\t' for start of the sentence and '\\n' to signify end of the sentence\n",
    "    spa_line = '\\t' + str(lines[line]).split('\\t')[1] + '\\n'\n",
    "    eng_sent.append(eng_line)\n",
    "    spa_sent.append(spa_line)\n",
    "    \n",
    "    for ch in eng_line:\n",
    "        if (ch not in eng_chars):\n",
    "            eng_chars.add(ch)\n",
    "            \n",
    "    for ch in spa_line:\n",
    "        if (ch not in spa_chars):\n",
    "            spa_chars.add(ch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Go.'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eng_sent[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\tVe.\\n'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spa_sent[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing of the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "spa_chars = sorted(list(spa_chars))\n",
    "eng_chars = sorted(list(eng_chars))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[' ', '!', '\"', '$', \"'\", ',', '-', '.', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', '?', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']\n"
     ]
    }
   ],
   "source": [
    "print(eng_chars)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['\\t', '\\n', ' ', '!', '\"', \"'\", ',', '-', '.', '0', '1', '2', '3', '4', '5', '6', '7', '8', ':', '?', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '¡', '«', '»', '¿', 'Á', 'É', 'Ó', 'Ú', 'á', 'é', 'í', 'ñ', 'ó', 'ú', 'ü']\n"
     ]
    }
   ],
   "source": [
    "print(spa_chars)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dictionary to index each english character - key is index and value is english character\n",
    "eng_id2char = {}\n",
    "\n",
    "# dictionary to get english character given its index - key is english character and value is index\n",
    "eng_char2id = {}\n",
    "\n",
    "for i, j in enumerate(eng_chars):\n",
    "    eng_id2char[i] = j\n",
    "    eng_char2id[j] = i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{0: ' ', 1: '!', 2: '\"', 3: '$', 4: \"'\", 5: ',', 6: '-', 7: '.', 8: '0', 9: '1', 10: '2', 11: '3', 12: '4', 13: '5', 14: '6', 15: '7', 16: '8', 17: '9', 18: ':', 19: '?', 20: 'A', 21: 'B', 22: 'C', 23: 'D', 24: 'E', 25: 'F', 26: 'G', 27: 'H', 28: 'I', 29: 'J', 30: 'K', 31: 'L', 32: 'M', 33: 'N', 34: 'O', 35: 'P', 36: 'Q', 37: 'R', 38: 'S', 39: 'T', 40: 'U', 41: 'V', 42: 'W', 43: 'Y', 44: 'Z', 45: 'a', 46: 'b', 47: 'c', 48: 'd', 49: 'e', 50: 'f', 51: 'g', 52: 'h', 53: 'i', 54: 'j', 55: 'k', 56: 'l', 57: 'm', 58: 'n', 59: 'o', 60: 'p', 61: 'q', 62: 'r', 63: 's', 64: 't', 65: 'u', 66: 'v', 67: 'w', 68: 'x', 69: 'y', 70: 'z'}\n"
     ]
    }
   ],
   "source": [
    "print(eng_id2char)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dictionary to index each english character - key is index and value is english character\n",
    "spa_id2char = {}\n",
    "\n",
    "# dictionary to get english character given its index - key is english character and value is index\n",
    "spa_char2id = {}\n",
    "\n",
    "for i, j in enumerate(spa_chars):\n",
    "    spa_id2char[i] = j\n",
    "    spa_char2id[j] = i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max_len_eng = max([len(line) for line in eng_sent])\n",
    "max_len_eng"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max_len_spa = max([len(line) for line in spa_sent])\n",
    "max_len_spa"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tokenizers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenized_eng_sent = np.zeros(shape = (n_samples,max_len_eng,len(eng_chars)), dtype='float32')\n",
    "tokenized_spa_sent = np.zeros(shape = (n_samples,max_len_spa,len(spa_chars)), dtype='float32')\n",
    "target_data = np.zeros((n_samples, max_len_spa, len(spa_chars)),dtype='float32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(n_samples):\n",
    "    for k,ch in enumerate(eng_sent[i]):\n",
    "        tokenized_eng_sent[i,k,eng_char2id[ch]] = 1\n",
    "        \n",
    "    for k,ch in enumerate(spa_sent[i]):\n",
    "        tokenized_spa_sent[i,k,spa_char2id[ch]] = 1\n",
    "\n",
    "        # decoder_target_data will be ahead by one timestep and will not include the start character.\n",
    "        if k > 0:\n",
    "            target_data[i,k-1,spa_char2id[ch]] = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 17, 71)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenized_eng_sent.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 42, 86)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenized_spa_sent.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 42, 86)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Encoder Model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "encoder_input = Input(shape=(None,len(eng_chars)))\n",
    "encoder_LSTM = CuDNNLSTM(256,return_state = True)\n",
    "encoder_outputs, encoder_h, encoder_c = encoder_LSTM (encoder_input)\n",
    "encoder_states = [encoder_h, encoder_c]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decoder model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "decoder_input = Input(shape=(None,len(spa_chars)))\n",
    "decoder_LSTM = CuDNNLSTM(256,return_sequences=True, return_state = True)\n",
    "decoder_out, _ , _ = decoder_LSTM(decoder_input, initial_state=encoder_states)\n",
    "decoder_dense = Dense(len(spa_chars),activation='softmax')\n",
    "decoder_out = decoder_dense(decoder_out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Model(inputs=[encoder_input, decoder_input],outputs=[decoder_out])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_1 (InputLayer)            (None, None, 71)     0                                            \n",
      "__________________________________________________________________________________________________\n",
      "input_2 (InputLayer)            (None, None, 86)     0                                            \n",
      "__________________________________________________________________________________________________\n",
      "cu_dnnlstm_1 (CuDNNLSTM)        [(None, 256), (None, 336896      input_1[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "cu_dnnlstm_2 (CuDNNLSTM)        [(None, None, 256),  352256      input_2[0][0]                    \n",
      "                                                                 cu_dnnlstm_1[0][1]               \n",
      "                                                                 cu_dnnlstm_1[0][2]               \n",
      "__________________________________________________________________________________________________\n",
      "dense_1 (Dense)                 (None, None, 86)     22102       cu_dnnlstm_2[0][0]               \n",
      "==================================================================================================\n",
      "Total params: 711,254\n",
      "Trainable params: 711,254\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 8000 samples, validate on 2000 samples\n",
      "Epoch 1/20\n",
      "8000/8000 [==============================] - 10s 1ms/step - loss: 1.1324 - acc: 0.0800 - val_loss: 1.1885 - val_acc: 0.1315\n",
      "Epoch 2/20\n",
      "8000/8000 [==============================] - 2s 193us/step - loss: 0.9359 - acc: 0.1245 - val_loss: 1.0227 - val_acc: 0.1571\n",
      "Epoch 3/20\n",
      "8000/8000 [==============================] - 1s 187us/step - loss: 0.8193 - acc: 0.1458 - val_loss: 0.9329 - val_acc: 0.1706\n",
      "Epoch 4/20\n",
      "8000/8000 [==============================] - 2s 188us/step - loss: 0.7540 - acc: 0.1627 - val_loss: 0.8822 - val_acc: 0.1846\n",
      "Epoch 5/20\n",
      "8000/8000 [==============================] - 1s 187us/step - loss: 0.7035 - acc: 0.1770 - val_loss: 0.8284 - val_acc: 0.1986\n",
      "Epoch 6/20\n",
      "8000/8000 [==============================] - 1s 187us/step - loss: 0.6624 - acc: 0.1883 - val_loss: 0.7898 - val_acc: 0.2121\n",
      "Epoch 7/20\n",
      "8000/8000 [==============================] - 1s 187us/step - loss: 0.6276 - acc: 0.1994 - val_loss: 0.7706 - val_acc: 0.2167\n",
      "Epoch 8/20\n",
      "8000/8000 [==============================] - 2s 188us/step - loss: 0.5968 - acc: 0.2091 - val_loss: 0.7330 - val_acc: 0.2305\n",
      "Epoch 9/20\n",
      "8000/8000 [==============================] - 1s 187us/step - loss: 0.5701 - acc: 0.2172 - val_loss: 0.7183 - val_acc: 0.2354\n",
      "Epoch 10/20\n",
      "8000/8000 [==============================] - 2s 189us/step - loss: 0.5468 - acc: 0.2242 - val_loss: 0.7072 - val_acc: 0.2371\n",
      "Epoch 11/20\n",
      "8000/8000 [==============================] - 2s 188us/step - loss: 0.5261 - acc: 0.2308 - val_loss: 0.6849 - val_acc: 0.2456\n",
      "Epoch 12/20\n",
      "8000/8000 [==============================] - 2s 190us/step - loss: 0.5063 - acc: 0.2358 - val_loss: 0.6799 - val_acc: 0.2481\n",
      "Epoch 13/20\n",
      "8000/8000 [==============================] - 2s 188us/step - loss: 0.4882 - acc: 0.2418 - val_loss: 0.6712 - val_acc: 0.2504\n",
      "Epoch 14/20\n",
      "8000/8000 [==============================] - 2s 192us/step - loss: 0.4717 - acc: 0.2467 - val_loss: 0.6611 - val_acc: 0.2533\n",
      "Epoch 15/20\n",
      "8000/8000 [==============================] - 2s 190us/step - loss: 0.4558 - acc: 0.2516 - val_loss: 0.6517 - val_acc: 0.2568\n",
      "Epoch 16/20\n",
      "8000/8000 [==============================] - 2s 189us/step - loss: 0.4411 - acc: 0.2554 - val_loss: 0.6525 - val_acc: 0.2583\n",
      "Epoch 17/20\n",
      "8000/8000 [==============================] - 2s 192us/step - loss: 0.4270 - acc: 0.2602 - val_loss: 0.6454 - val_acc: 0.2605\n",
      "Epoch 18/20\n",
      "8000/8000 [==============================] - 1s 186us/step - loss: 0.4130 - acc: 0.2640 - val_loss: 0.6454 - val_acc: 0.2613\n",
      "Epoch 19/20\n",
      "8000/8000 [==============================] - 2s 188us/step - loss: 0.4005 - acc: 0.2679 - val_loss: 0.6373 - val_acc: 0.2642\n",
      "Epoch 20/20\n",
      "8000/8000 [==============================] - 2s 188us/step - loss: 0.3878 - acc: 0.2720 - val_loss: 0.6346 - val_acc: 0.2659\n",
      "Wall time: 39.4 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x1d6116715f8>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "model.fit(x=[tokenized_eng_sent,tokenized_spa_sent], \n",
    "          y=target_data,\n",
    "          batch_size=64,\n",
    "          epochs=20,\n",
    "          validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inference model for Testing:\n",
    "\n",
    "1) Encode input and retrieve initial decoder state.\n",
    "\n",
    "2) Run one step of decoder with this initial state\n",
    "   and a \"start of sequence\" token as target.\n",
    "   Output will be the next target token.\n",
    "\n",
    "3) Repeat with the current target token and current states."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Encoder inference model\n",
    "encoder_model_inf = Model(encoder_input, encoder_states)\n",
    "\n",
    "# Decoder inference model\n",
    "decoder_state_input_h = Input(shape=(256,))\n",
    "decoder_state_input_c = Input(shape=(256,))\n",
    "decoder_input_states = [decoder_state_input_h, decoder_state_input_c]\n",
    "\n",
    "decoder_out, decoder_h, decoder_c = decoder_LSTM(decoder_input, \n",
    "                                                 initial_state=decoder_input_states)\n",
    "\n",
    "decoder_states = [decoder_h , decoder_c]\n",
    "\n",
    "decoder_out = decoder_dense(decoder_out)\n",
    "\n",
    "decoder_model_inf = Model(inputs=[decoder_input] + decoder_input_states, outputs=[decoder_out] + decoder_states)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "def decode_seq(inp_seq):\n",
    "    \n",
    "    # Initial states value is coming from the encoder \n",
    "    states_val = encoder_model_inf.predict(inp_seq)\n",
    "    \n",
    "    target_seq = np.zeros((1, 1, len(spa_chars)))\n",
    "    target_seq[0, 0, spa_char2id['\\t']] = 1\n",
    "    \n",
    "    translated_sent = ''\n",
    "    stop_condition = False\n",
    "    \n",
    "    while not stop_condition:   \n",
    "        decoder_out, decoder_h, decoder_c = decoder_model_inf.predict(x=[target_seq] + states_val)\n",
    "        \n",
    "        max_val_index = np.argmax(decoder_out[0,-1,:])\n",
    "        sampled_spa_char = spa_id2char[max_val_index]\n",
    "        translated_sent += sampled_spa_char\n",
    "        \n",
    "        if ((sampled_spa_char == '\\n') or (len(translated_sent) > max_len_spa)) :\n",
    "            stop_condition = True\n",
    "        \n",
    "        target_seq = np.zeros((1, 1, len(spa_chars)))\n",
    "        target_seq[0, 0, max_val_index] = 1\n",
    "        \n",
    "        states_val = [decoder_h, decoder_c]\n",
    "        \n",
    "    return translated_sent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------\n",
      "Input sentence: Go.\n",
      "Decoded sentence: Vete.\n",
      "\n",
      "--------\n",
      "Input sentence: Fire!\n",
      "Decoded sentence: ¡Pueda esto!\n",
      "\n",
      "--------\n",
      "Input sentence: Wait!\n",
      "Decoded sentence: ¡Espera!\n",
      "\n",
      "--------\n",
      "Input sentence: Relax.\n",
      "Decoded sentence: Larmate.\n",
      "\n",
      "--------\n",
      "Input sentence: Hop in.\n",
      "Decoded sentence: ¡Estraba en las!\n",
      "\n",
      "--------\n",
      "Input sentence: I'm 19.\n",
      "Decoded sentence: Estoy llena.\n",
      "\n",
      "--------\n",
      "Input sentence: No way!\n",
      "Decoded sentence: ¡No puede ser!\n",
      "\n",
      "--------\n",
      "Input sentence: We try.\n",
      "Decoded sentence: Lo hamos misto.\n",
      "\n",
      "--------\n",
      "Input sentence: Beat it.\n",
      "Decoded sentence: Tramete.\n",
      "\n",
      "--------\n",
      "Input sentence: Come on.\n",
      "Decoded sentence: Van a come.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "values_idx = [0, 10, 20, 30, 40, 50, 60, 70, 80, 90]\n",
    "for index in values_idx:\n",
    "    inp_seq = tokenized_eng_sent[index:index+1]\n",
    "    translated_sent = decode_seq(inp_seq)\n",
    "    print('--------')\n",
    "    print('Input sentence:', eng_sent[index])\n",
    "    print('Decoded sentence:', translated_sent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reference:\n",
    "#### Sentence pairs: http://www.manythings.org/anki/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tf1.11_gpu",
   "language": "python",
   "name": "tf111_gpu"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}