Thing Translator

Article Source

Introduction

In Part I of my tutorial we built a Web API that handles image labeling and translation into different languages. Now we will build Android app that can consume that API. Features we are going to have are:
  • select destination language
  • select image from gallery or take new photo using camera
  • option to toogle speech output on/off

Using the code

All the code base is located in code section of this article. For simplicity reasons here I am going to highlight only the important parts of the code.
build.gradle
dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    compile 'com.google.code.gson:gson:2.6.1'
    compile 'com.android.support:support-v13:23.4.0'
    compile 'com.android.support:appcompat-v7:23.4.0'
    compile 'com.android.support:design:23.4.0'
    compile 'com.squareup.retrofit2:retrofit:2.0.0'
    compile 'com.squareup.retrofit2:converter-gson:2.0.0'
    compile 'com.squareup.okhttp3:okhttp:3.4.1'
    compile 'com.android.support.constraint:constraint-layout:1.0.0-alpha4'
    testCompile 'junit:junit:4.12'
    compile 'com.pixplicity.easyprefs:library:1.8.1@aar'
    compile 'com.jakewharton:butterknife:8.4.0'
    annotationProcessor 'com.jakewharton:butterknife-compiler:8.4.0'
}
We will use some really cool Java libraries that makes Android development lot easier.
  • easyprefs -  easy access to SharedPrefferences
  • butterknife - makes Views declaration lot easier
  • retrofit - enables us to work with REST API's quickly and easily 
activity_main - spinner View has a custom layout defined in spinner_item.xml
<Spinner
    android:id="@+id/spinner"
    android:layout_width="100dp"
    android:layout_height="match_parent"
    android:layout_marginBottom="16dp"
    android:layout_weight="0.91"
    android:gravity="bottom|right" />

<?xml version="1.0" encoding="utf-8"?>
<TextView
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="wrap_content"
    android:layout_height="30dp"
    android:textColor="@android:color/darker_gray"
    android:textSize="24dp"
    />
ApiInterface.java - this is interface for our REST API calls. Put your API url we build in previous tutorial into ENDPOINT variable. As you can see only 2 parameters are used when making POST request:
  1. Image file
  2. Language Code
package thingtranslator2.jalle.com.thingtranslator2.Rest;
 

public interface ApiInterface {
    
    String ENDPOINT = "YOUR-API-URL/api/";

    @Multipart
    @POST("upload")
    Call<translation> upload(@Part("image\"; filename=\"pic.jpg\" ") RequestBody file, @Part("FirstName") RequestBody langCode1);

    final OkHttpClient okHttpClient = new OkHttpClient.Builder()
            .readTimeout(60, TimeUnit.SECONDS)
            .connectTimeout(60, TimeUnit.SECONDS)
            .build();

    Retrofit retrofit = new Retrofit.Builder()
            .baseUrl(ENDPOINT)
            .client(okHttpClient)
            .addConverterFactory(GsonConverterFactory.create())
            .build();
}

Translation class
package thingtranslator2.jalle.com.thingtranslator2.Rest;

public class Translation {
    public String Original;
    public String Translation;
    public String Error;
}
Tools - ImagePicker this class is used for getting image from gallery of taking new photo from camera. Tools - MarshMallowPermission this is used for setting up camera permissions on Android 6.
AndroidManifest
    <?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="thingtranslator2.jalle.com.thingtranslator2">

    <uses-feature android:name="android.hardware.camera" />
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-feature android:name="android.hardware.camera.autofocus" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity" >

            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>
</manifest>
MainActivity.java
public class MainActivity extends AppCompatActivity implements AdapterView.OnItemSelectedListener {
    public
    @BindView(R.id.txtTranslation)
    TextView txtTranslation;
    public
    @BindView(R.id.spinner)
    Spinner spinner;
    public
    @BindView(R.id.imgPhoto)
    ImageButton imgPhoto;
    public
    @BindView(R.id.imgSpeaker)
    ImageButton btnSpeaker;

    public Boolean speakerOn, firstRun;
    public TextToSpeech tts;
    public List<string> languages = new ArrayList<string>();
    public ArrayList<language> languageList = new ArrayList<language>();
    public ArrayAdapter<language> spinnerArrayAdapter;
    public Language selectedLanguage;
    public static final int PICK_IMAGE_ID = 234; // the number doesn't matter
    thingtranslator2.jalle.com.thingtranslator2.MarshMallowPermission marshMallowPermission = new thingtranslator2.jalle.com.thingtranslator2.MarshMallowPermission(this);

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN);
        ButterKnife.bind(this);

        //init our Shared preferences helper
        new Prefs.Builder()
                .setContext(this)
                .setMode(ContextWrapper.MODE_PRIVATE)
                .setPrefsName(getPackageName())
                .setUseDefaultSharedPreference(true)
                .build();

        //  Prefs.clear();
        firstRun = Prefs.getBoolean("firstRun", true);
        spinner.setOnItemSelectedListener(this);
        tts = new TextToSpeech(getApplicationContext(), new TextToSpeech.OnInitListener() {
            @Override
            public void onInit(int status) {
                if (status != TextToSpeech.ERROR) {
                    fillSpinner();


                }
            }
        });


        if (firstRun) {
            Prefs.putBoolean("firstRun", false);
            Prefs.putBoolean("speakerOn", true);
            Prefs.putString("langCode", "bs");
        }
        setSpeaker();
    }

    private void setSpeaker() {
        int id = Prefs.getBoolean("speakerOn", false) ? R.drawable.ic_volume : R.drawable.ic_volume_off;
        btnSpeaker.setImageBitmap(BitmapFactory.decodeResource(getResources(), id));
    }

    public void ToogleSpeaker(View v) {
        Prefs.putBoolean("speakerOn", !Prefs.getBoolean("speakerOn", false));
        setSpeaker();
        Toast.makeText(getApplicationContext(), "Speech " + (Prefs.getBoolean("speakerOn", true) ? "On" : "Off"), Toast.LENGTH_SHORT);
    }

    private void fillSpinner() {
        Iterator itr = tts.getAvailableLanguages().iterator();
        while (itr.hasNext()) {
            Locale item = (Locale) itr.next();
            languageList.add(new Language(item.getDisplayName(), item.getLanguage()));
        }
        //Sort that array
        Collections.sort(languageList, new Comparator<language>() {
            @Override
            public int compare(Language o1, Language o2) {
                return o1.getlangCode().compareTo(o2.getlangCode());
            }
        });

        spinnerArrayAdapter = new ArrayAdapter<language>(this, R.layout.spinner_item, languageList);
        spinnerArrayAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
        spinner.setAdapter(spinnerArrayAdapter);

        //Set the default language to bs- Bosnian
        String def = Prefs.getString("langCode", "bs");
        for (Language lang : languageList) {
            if (lang.getlangCode().equals(def)) {
                spinner.setSelection(languageList.indexOf(lang));
            }
        }
    }

    ;

    //Get image from  Gallery or Camera
    public void getImage(View v) {
        if (!marshMallowPermission.checkPermissionForCamera()) {
            marshMallowPermission.requestPermissionForCamera();
        } else {
            if (!marshMallowPermission.checkPermissionForExternalStorage()) {
                marshMallowPermission.requestPermissionForExternalStorage();
            } else {
                Intent chooseImageIntent = thingtranslator2.jalle.com.thingtranslator2.ImagePicker.getPickImageIntent(getApplicationContext());
                startActivityForResult(chooseImageIntent, PICK_IMAGE_ID);
            }
        }
    }


    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (data == null) return;

        switch (requestCode) {
            case PICK_IMAGE_ID:
                Bitmap bitmap = thingtranslator2.jalle.com.thingtranslator2.ImagePicker.getImageFromResult(this, resultCode, data);
                imgPhoto.setImageBitmap(null);
                imgPhoto.setBackground(null);
                imgPhoto.setImageBitmap(bitmap);
                imgPhoto.invalidate();
                imgPhoto.postInvalidate();

                File file = null;
                try {
                    file = savebitmap(bitmap, "pic.jpeg");
                } catch (IOException e) {
                    e.printStackTrace();
                }
                uploadImage(file);
                break;
            default:
                super.onActivityResult(requestCode, resultCode, data);
                break;
        }
    }

    private void uploadImage(File file) {
        RequestBody body = RequestBody.create(MediaType.parse("image/*"), file);
        RequestBody langCode1 = RequestBody.create(MediaType.parse("text/plain"), selectedLanguage.langCode);

        final ProgressDialog progress = new ProgressDialog(this);
        progress.setMessage("Processing image...");
        progress.setProgressStyle(ProgressDialog.STYLE_SPINNER);
        progress.setIndeterminate(true);
        progress.show();


        ApiInterface mApiService = ApiInterface.retrofit.create(ApiInterface.class);
        Call<translation> mService = mApiService.upload(body, langCode1);
        mService.enqueue(new Callback<translation>() {
            @Override
            public void onResponse(Call<translation> call, Response<translation> response) {
                progress.hide();
                Translation result = response.body();
                txtTranslation.setText(result.Translation);
                txtTranslation.invalidate();

                if (Prefs.getBoolean("speakerOn", true)) {
                    tts.speak(result.Translation, TextToSpeech.QUEUE_FLUSH, null);
                }
            }

            @Override
            public void onFailure(Call<translation> call, Throwable t) {
                call.cancel();
                progress.hide();
                Toast.makeText(getApplicationContext(), "Error: " + t.getMessage(), Toast.LENGTH_LONG).show();
            }
        });
    }


    public static File savebitmap(Bitmap bmp, String fName) throws IOException {
        ByteArrayOutputStream bytes = new ByteArrayOutputStream();
        bmp.compress(Bitmap.CompressFormat.JPEG, 60, bytes);
        File f = new File(Environment.getExternalStorageDirectory()
                + File.separator + fName);
        f.createNewFile();
        FileOutputStream fo = new FileOutputStream(f);
        fo.write(bytes.toByteArray());
        fo.close();
        return f;
    }


    @Override
    public void onItemSelected(AdapterView parent, View view, int position, long id) {

        selectedLanguage = (Language) spinner.getSelectedItem();
        //  langCode = languages.get(position);
        tts.setLanguage(Locale.forLanguageTag(selectedLanguage.langCode));
        Prefs.putString("langCode", selectedLanguage.langCode);
    }

    @Override
    public void onNothingSelected(AdapterView parent) {

    }
}
The end result of this tutorial is an application that looks like this. Remember your WebAPI service needs to be up and running in order for all this to work.

Points of Interest

There are many ways for  improvement and upgrading this application. One of my ideas is that the image labelling could be done during the live preview on your camera. Having the preview window opened, we can invoke the API every 5-10 seconds and have live labelling about what we see through our camera.
One drawback is that  it would consume a lot of bandwidth and also remember, Google API Vision and most of the other APIs are not free. You pay for the API usage when requests exceed the allowed quota.
Source code available at Github

Comments

Post a Comment

Popular posts from this blog

Building NodeJS REST service and consuming it from within Android application

Image
Building NodeJS REST service and consuming it from within Android application Introduction In this tutorial I will walk you through the process of building REST service in NodeJS. That service will enable Android users to authenticate to the back-end and set their profile photo. Using the code The application consist of two parts: Backend: RESTful service written in NodeJS Mobile: Android app written in Java Part 1. - NodeJS back-end Back-end part will enable users to: Login using email/password combination Upload their profile picture The source code is located on Github . For simplicity reasons here I am going to highlight only the important parts of the code. app.js var express = require('express'); var http = require('http'); var path = require('path'); var app = express(); var util = require('util'); var multiparty = require('multiparty'); var multer = require('multer'); var upload = multer({ dest:
Read more

Building ASP.NET Core app strengthened with AngularJS 2

Image
Article Source Introduction In this tutorial I will show you how to create Master-Detail web application that also have a search option. We will use the data of the Cruising Company. It contains data about: ships, booking info, sale units etc. Using the code The application consist of two parts: Backend: RESTful API written in .NET Core Frontend: SPA written in Angular 2 Download and install following tools and libraries: .NET Core 1.0.1 TypeScript 2.0 Node.js v4.0 or above. ASP.NET Core Template Pack visz file The source code is located on Github . For simplicity reasons here I am going to highlight only the important parts of the code. Start by creating new project: New -> Project. Select Web -> ASP.NET Core Angular 2 Starter. Replace following files with the appropriate code. project.json { "dependencies": { "Microsoft.NETCore.App": { "version": "1.0.1", "type": "
Read more