mary.codes

Creating a Duolingo Widget in React and PHP

Blog icon
7 min read

Programming

Attempting to create a widget to display my Duolingo progress with the Duolingo API, React, and some back-end PHP.

Update: The Duolingo API endpoint has changed and no longer works as documented below.

I've been incrementally adding things to the homepage of my site that keep track of what I'm up to. I have a new section for the current book that I'm reading and audiobook that I'm listening to, which is updated manually for right now because, well, print books don't have an API and Audible doesn't have a public API. (Shame!)

I've picked up Duolingo again and thought it would be nice to add my progress as a section of my homepage. Duolingo also doesn't have an official, supported API, but the site tschuy.com maps out a few of the public endpoints.

React

I'm creating this widget as a separate component within the src/components directory so I can just plug it into the correct spot in my homepage with <DuolingoProgress/>. Also, this is my first time using React's new Hooks system!

import React, {useState, useEffect} from 'react';
import axios from 'axios';

const DuolingoProgress = () => {
  const [languagedata, setLanguageData] = useState([]);
  const [streak, setStreak] = useState(0);

  useEffect(() => {
    axios.get('https://www.duolingo.com/api/1/users/show?id=MY_USER_ID').then((response) => {
      const data = response.data;
      const currentlangs = data.languages.filter(l => {
        if (l.learning) {
          return l;
        } else {
          return false;
        }
      });
      setLanguageData(currentlangs);
      setStreak(data.site_streak);
    }).catch((err) => {
      console.log(err);
    })
  }, []);

  return (
    <div>
      <div>{streak} day Duolingo streak</div>
      {languagedata.map((lang, i) => {
        return (
          <div key={`language${i}`} className="languages">
            <div className="languagename" key={lang.language_string}>{lang.language_string}</div>
            <div key={lang.level}>Level: {lang.level}</div>
            <div key={lang.points}>Points: {lang.points}</div>
          </div>
        )
      })}
    </div>
  );
}

export default DuolingoProgress;

I figured that this would be a simple way to learn how to use Hooks. As you can see, instead of creating a class for my new component, I'm just creating a function component and importing useState and useEffect from React.

Instead of using a constructor inside of a class, we can simply declare a state variable using useState. In this case, I'm declaring "language" and "streak" as two variables that I'm going to set from the Duolingo API and I'm setting their initial values with useState. UseState returns both the variable and a function that updates the variable, which is why we also include the function names "setLanguageData" and "setStreak".

  const [languagedata, setLanguageData] = useState([]);
  const [streak, setStreak] = useState(0);

UseEffect is another React hook that allows us to perform side effects in function components. In a React class I would put the API call into componentDidMount, but useEffect replaces that, as well as componentDidUpdate and componentWillUnmount.

UseEffect runs after every page render, which is not optimal for an API call. I don't want my page constantly hitting the API endpoint, begging for an update and hurting page performance. The Effect Hook allows you to pass an argument that specifies when useEffect should run. In our case, we want to run our API call only once, so we pass in an empty array.

Also notice that once we get our data from the API, we set our state variables using their update functions, "setLanguageData" and "setStreak";

  useEffect(() => {
    axios.get('https://www.duolingo.com/api/1/users/show?id=MY_USER_ID').then((response) => {
      const data = response.data;
      const currentlangs = data.languages.filter(l => {
        if (l.learning) {
          return l;
        } else {
          return false;
        }
      });
      setLanguageData(currentlangs);
      setStreak(data.site_streak);
    }).catch((err) => {
      console.log(err);
    })
  }, []);

Lastly, we return and render the widget. I'm mapping over each language I'm learning and returning my current level and points. Notice that using state variables means we can use the variable name directly, for example {languagedata}, instead of the old way of {this.state.languagedata}.

  return (
    <div>
      <div>{streak} day Duolingo streak</div>
      {languagedata.map((lang, i) => {
        return (
          <div key={`language${i}`} className="languages">
            <div className="languagename" key={lang.language_string}>{lang.language_string}</div>
            <div key={lang.level}>Level: {lang.level}</div>
            <div key={lang.points}>Points: {lang.points}</div>
          </div>
        )
      })}
    </div>
  );

Running the page with gatsby develop gives us this (with a tiny bit of styling):

Language learning widget

However, there is a problem with the Duolingo API.

This was working just fine on localhost:8000, but if I change the URL to 127.0.0.1:8000 or try to build and deploy...

CORS header

Curses!

One workaround would be to use CORS-Anywhere.

axios.get('https://cors-anywhere.herokuapp.com/https://www.duolingo.com/api/1/users/show?id=MY_USER_ID')

Now it does work if I try to access the development site through 127.0.0.1:8000.

Here's my problem, though. I don't want to rely on CORS-Anywhere to always be around. If it went offline for any reason, the widget would break. I could fork CORS-Anywhere and host it myself. There's also an unofficial Duolingo API written in Python. However, what I think I'm going to do is use PHP and cURL to get the information I want and host it on my projects server.

PHP

In a separate PHP file (and directory, I might add), I whipped up a quick cURL script to grab the API data and tested it with a quick and dirty PHP server on the command line: php -S 127.0.0.1:8080.

$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, 'https://www.duolingo.com/api/1/users/show?id=MY_USER_ID');
$result = curl_exec($curl);
curl_close($curl);
print_r($result);

If I visit 127.0.0.1:8080/api.php, I'll see the API info printed out to the screen.

Now, I need a way to cache this information. This isn't a step that I necessarily need. It would probably be just fine to have my server fetch the freshest information each time the homepage is visited. However, caching the information would be faster for subsequent visits, my Duolingo progress only changes a few times a day, and I want some practice with server-side caching. For these reasons, I've decided to use Memcached. First, I have to install Memcached and restart Apache.

sudo apt-get install memcached
sudo apt-get install php-memcache
sudo service memcached start
sudo service apache2 restart

Now, I update my code to use php-memcache and only cache the information I need. I added a 300 second (5 minute) expiration on the cached data. I also added a CORS header to my file so I can access it from my locally-run Gatsby instance. Yes, trying to access 127.0.0.1:8080 from 127.0.0.1:8000 will still fail!

header('Access-Control-Allow-Origin: http://127.0.0.1:8000');
header('Content-Type: application/json');

$memcache_host = '127.0.0.1';
$memcache_port = 11211;

$memcache = new Memcache;
$memcache->connect($memcache_host, $memcache_port) or die ("Could not connect");

$duolingo_data = $memcache->get('duolingo_data');

if (!$duolingo_data) {
    $curl = curl_init();
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($curl, CURLOPT_URL, "https://www.duolingo.com/api/1/users/show?id=MY_USER_ID");
    $result = curl_exec($curl);
    curl_close($curl);

    $data = json_decode($result, true);
    $current_languages = array_filter($data['languages'], function($lang) {
        if ($lang['learning']) {
            return $lang;
        }
    });
    $duolingo_data['username'] = $data['username'];
    $duolingo_data['site_streak'] = $data['site_streak'];
    $duolingo_data['current_languages'] = $current_languages;

    $memcache->set('duolingo_data', $duolingo_data, false, 300);
}

echo json_encode($duolingo_data, JSON_PRETTY_PRINT);

To test the cache, I accessed the page at 127.0.0.1:8080/api.php with my 3-day Duolingo streak still in place, then quickly visited Duolingo and finished a lesson to increase my streak. I refreshed the page and it quickly loaded the cached version. After a few more minutes, I refreshed the page again. It loaded the updated information. My five-minute cache works!

Duolingo API

Side note: This is how I manually cleared the cache on the command line during testing instead of waiting the full five minutes.

$ telnet 127.0.0.1 11211
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
flush_all
OK
quit
Connection closed by foreign host.

The next challenge was getting the script onto my projects server, which was a nightmare in and of itself. I'm not going to go into that in detail, but I finally got it working. In short: I'm still not great with Nginx, or at least not as comfortable with Nginx server blocks as I am with Apache virtual hosts.

Finally, let's circle back around to the React part of the equation. I made a few adjustments to accommodate some of the changes in the API.

React refactor

import React, {useState, useEffect} from 'react';
import axios from 'axios';

const DuolingoProgress = () => {
  const [username, setUsername] = useState(null);
  const [languagedata, setLanguageData] = useState([]);
  const [streak, setStreak] = useState(0);

  useEffect(() => {
    axios.get('https://apis.maryknize.com/duolingo_api.php').then((response) => {
      const data = response.data;
      const currentlangs = Object.values(data.current_languages).map(l => {
        return l;
      });
      currentlangs.sort((a, b) => {
        return b.points - a.points;
      });
      setUsername(data.username);
      setLanguageData(currentlangs);
      setStreak(data.site_streak);
    }).catch((err) => {
      console.log(err);
    })
  }, []);

  return (
    <div>
      <p><a href={`https://www.duolingo.com/${username}`}>{username}</a>: {streak}-day streak</p>
      <div style={{textAlign: `center`}}>
        {languagedata.map((lang, i) => {
          let langclass;
          if (lang.current_learning) {
            langclass = "languages_current";
          } else {
            langclass = "languages";
          }

          return (
            <div key={`language${i}`} className={langclass}>
              <div className="languagename" key={lang.language_string}>{lang.language_string}</div>
              <div key={lang.level}>Current Level: {lang.level}</div>
              <div key={lang.points}>Total Points: {lang.points}</div>
              <div key={lang.to_next_level}>Level up in {lang.to_next_level} points.</div>
            </div>
          )
        })}
      </div>
      <p>Active language is outlined in <span style={{color: `#2E9CCA`}}>blue</span></p>
    </div>
  );
}

export default DuolingoProgress;

I kept most of my React component the same, replacing the earlier Duolingo API with my personal one. Since my PHP code only returns languages I'm currently learning, I can skip that process in the API response. I've also added in a check in my render response to see which language I have set as active, and then applying different styling to that language.

Here is my final result!

Duolingo widget