If you come across any mistakes or bugs in this tutorial, please let us know using a Github issue or a post on the DJI forum. Please feel free to send us Github pull request and help us fix any issues.
This tutorial is designed for you to gain a basic understanding of the DJI Mobile SDK. It will implement the FPV view and two basic camera functionalities: Take Photo and Record video.
You can download the tutorial's final sample project from this Github Page.
Application Activation and Aircraft Binding in China
For DJI SDK mobile application used in China, it's required to activate the application and bind the aircraft to the user's DJI account.
If an application is not activated, the aircraft not bound (if required), or a legacy version of the SDK (< 4.1) is being used, all camera live streams will be disabled, and flight will be limited to a zone of 100m diameter and 30m height to ensure the aircraft stays within line of sight.
Open Android Studio and select File -> New -> New Project to create a new project, named 'FPVDemo'. Enter the company domain and package name (Here we use "com.dji.FPVDemo") you want and press Next. Set the minimum SDK version as API 19: Android 4.4 (KitKat) for "Phone and Tablet" and press Next. Then select "Empty Activity" and press Next. Lastly, leave the Activity Name as "MainActivity", and the Layout Name as "activity_main", Press "Finish" to create the project.
In our previous tutorial Importing and Activating DJI SDK in Android Studio Project, you have learned how to import the Android SDK Maven Dependency and activate your application. If you haven't read that previously, please take a look at it and implement the related features. Once you've done that, continue to implement the next features.
Building the Layouts of Activity
1. Creating MApplication Class
Right click on the com.dji.FPVDemo, and select New->Java Class to create a new java class and name it as "MApplication".
Then, open the MApplication.java file and replace the content with the following:
Here we firstly override the attachBaseContext() method to invoke the install() method of Helper class to load the SDK classes before using any SDK functionality. Failing to do so will result in unexpected crashes. Next, override the onCreate() method to invoke the onCreate() method of FPVDemoApplication.
2. Creating FPVDemoApplication Class
Right-click on the package com.dji.FPVDemo in the project navigator and choose New -> Java Class, Type in "FPVDemoApplication" in the Name field and select "Class" as Kind field content.
Next, Replace the code of the "FPVDemoApplication.java" file with the following:
@Override publicvoidonClick(View v){ switch (v.getId()) { case R.id.btn_capture:{ break; } case R.id.btn_shoot_photo_mode:{ break; } case R.id.btn_record_video_mode:{ break; } default: break; } }
}
In the code shown above, we implement the following features:
1. Create the layout UI elements variables, including a TextureView mVideoSurface, three Buttons mCaptureBtn, mShootPhotoModeBtn, mRecordVideoModeBtn, one Toggle Button mRecordBtn and a TextView recordingTime.
2. Then invoke the initUI() method to initialize UI variables. And implement the setOnClickListener() method of Button for all the Buttons. Also implement the setOnCheckedChangeListener() method for Toggle Button.
3. Override the onClick() method to implement the three Buttons' click actions.
4. Implementing the MainActivity Layout
Open the activity_main.xml layout file and replace the code with the following:
In the xml file, we create a TextureView(id: video_previewer_surface) element to show the live video stream from the camera. Moreover, we implement a LinearLayout element to create the "Capture" Button(id: btn_capture), "Record" ToggleButton(id: btn_record), "Shoot Photo Mode" Button(id: btn_shoot_photo_mode) and "Record Video Mode" Button(id: btn_record_video_mode).
Lastly, we create a TextView(id: timer) element to show the record video time.
5. Implementing ConnectionActivity Class
To improve the user experience, we had better create an activity to show the connection status between the DJI Product and the SDK, once it's connected, the user can press the OPEN button to enter the MainActivity.
Now let's Right-click on the package com.dji.FPVDemo in the project navigator and choose New -> Activity -> Basic Activity, Type in "ConnectionActivity" in the "Activity Name" field and press "Finish" button.
Next, replace the code of the "ConnectionActivity.java" file with the following:
In the code shown above, we implement the following features:
Create the layout UI elements variables, including three TextureViews mTextConnectionStatus, mTextProduct, mVersionTv and one Button mBtnOpen.
In the onCreate() method, we invoke the initUI() methods to initialize the UI elements.
Next, implement the initUI() method to initialize the three TextViews and the Button. Then invoke setOnClickListener() method of mBtnOpen and pass this as the param.
Lastly, override the onClick() method to implement the Button's click action.
6. Implementing the ConnectionActivity Layout
Open the activity_connection.xml layout file and replace the code with the following:
In the xml file, we create four TextViews and one Button within a RelativeLayout. We use the TextView(id: text_connection_status) to show the product connection status and use the TextView(id:text_product_info) to show the connected product name. The Button(id: btn_open) is used to open the MainActivity.
7. Configuring the Resource XMLs
Once you finish the above steps, let's copy all the images file from this Github sample project's drawable folder (app->src->main->res->drawable) to the same folder in your project.
Moreover, open the "colors.xml" file and update the content as shown below:
Lastly, open the "styles.xml" file and replace the content with the followings:
<resources>
<!-- Base application theme. --> <stylename="AppTheme"parent="Theme.AppCompat.Light.DarkActionBar"> <!-- Customize your theme here. --> <itemname="colorPrimary">@color/colorPrimary</item> <itemname="colorPrimaryDark">@color/colorPrimaryDark</item> <itemname="colorAccent">@color/colorAccent</item> </style>
</resources>
Now, if you open the activity_main.xml file, and click on the Design tab on the bottom left, you should see the preview screenshots of MainActivity and ConnectionActivity as shown below:
ConnectionActivity
MainActivity
For more details, please check the Github source code of this tutorial.
Registering the Application
After you finish the above steps, let's register our application with the App Key you apply from DJI Developer Website. If you are not familiar with the App Key, please check the Get Started.
1. Let's open the AndroidManifest.xml file and add the following elements on top of the application element:
Here, we request permissions that the application must be granted in order for it to register DJI SDK correctly. Also, we declare the camera and USB hardwares which are used by the application.
Next, add the android:name=".MApplication" at the beginning of the application element:
In the code above, you should substitute your App Key of the application for "Please enter your App Key here." in the value attribute under the android:name="com.dji.sdk.API_KEY" attribute.
Lastly, update the "MainActivity" and "ConnectionActivity" activity elements as shown below:
In the code above, we add the attributes of "android:screenOrientation" to set "ConnectionActivity" as portrait and set "MainActivity" as landscape.
2. After you finish the steps above, open the "FPVDemoApplication.java" file and replace the code with the same file in the Github Source Code, here we explain the important parts of it:
@Override publicvoidonCreate(){ super.onCreate(); mHandler = new Handler(Looper.getMainLooper());
/** * When starting SDK services, an instance of interface DJISDKManager.DJISDKManagerCallback will be used to listen to * the SDK Registration result and the product changing. */ mDJISDKManagerCallback = new DJISDKManager.SDKManagerCallback() {
//Listens to the SDK registration result @Override publicvoidonRegister(DJIError error){
@Override publicvoidonInitProcess(DJISDKInitEvent djisdkInitEvent, int i){
}
@Override publicvoidonDatabaseDownloadProgress(long l, long l1){
} };
//Check the permissions before registering the application for android system 6.0 above. int permissionCheck = ContextCompat.checkSelfPermission(getApplicationContext(), android.Manifest.permission.WRITE_EXTERNAL_STORAGE); int permissionCheck2 = ContextCompat.checkSelfPermission(getApplicationContext(), android.Manifest.permission.READ_PHONE_STATE); if (Build.VERSION.SDK_INT < Build.VERSION_CODES.M || (permissionCheck == 0 && permissionCheck2 == 0)) { //This is used to start SDK services and initiate SDK. DJISDKManager.getInstance().registerApp(getApplicationContext(), mDJISDKManagerCallback); Toast.makeText(getApplicationContext(), "registering, pls wait...", Toast.LENGTH_LONG).show();
} else { Toast.makeText(getApplicationContext(), "Please check if the permission is granted.", Toast.LENGTH_LONG).show(); } }
Here, we implement several features:
We override the onCreate() method to initialize the mHandler and mDJISDKManagerCallback instance variables and implement their callback methods.
For the four interface methods of SDKManagerCallback. we use the onRegister() method to check the Application registration status and show text message here. When the product is connected or disconnected, the onProductConnect() and onProductDisconnect() methods will be invoked. Moreover, we use the onComponentChange() method to check the component changes and invoke the notifyStatusChange() method to notify the changes.
Check the permissions of WRITE_EXTERNAL_STORAGE and READ_PHONE_STATE, then invoke the registerApp() method of DJISDKManager to register the application.
Now let's build and run the project and install it to your Android device. If everything goes well, you should see the "Register Success" textView like the following screenshot when you register the app successfully.
Important: Please initialize the DJI Android SDK class objects inside the onCreate() method after the SDK classes are loaded, failing to do so will result in unexpected crashes.
Once you finish the steps above, let's open the "ConnectionActivity.java" file and create several variables for checking permission and registration above the onCreate() method:
/** * Checks if there is any missing permissions, and * requests runtime permission if needed. */ privatevoidcheckAndRequestPermissions(){ // Check for permissions for (String eachPermission : REQUIRED_PERMISSION_LIST) { if (ContextCompat.checkSelfPermission(this, eachPermission) != PackageManager.PERMISSION_GRANTED) { missingPermission.add(eachPermission); } } // Request for missing permissions if (!missingPermission.isEmpty() && Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) { ActivityCompat.requestPermissions(this, missingPermission.toArray(new String[missingPermission.size()]), REQUEST_PERMISSION_CODE); }
}
/** * Result of runtime permission request */ @Override publicvoidonRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults){ super.onRequestPermissionsResult(requestCode, permissions, grantResults); // Check for granted permission and remove from missing list if (requestCode == REQUEST_PERMISSION_CODE) { for (int i = grantResults.length - 1; i >= 0; i--) { if (grantResults[i] == PackageManager.PERMISSION_GRANTED) { missingPermission.remove(permissions[i]); } } } // If there is enough permission, we will start the registration if (missingPermission.isEmpty()) { startSDKRegistration(); } else { showToast("Missing permissions!!!"); } }
@Override publicvoidonInitProcess(DJISDKInitEvent djisdkInitEvent, int i){
}
@Override publicvoidonDatabaseDownloadProgress(long l, long l1){
}
}); } }); } }
In the code shown above, we implement the following features:
In the onCreate() method, we invoke the checkAndRequestPermissions() method to check if there is any missing permissions, and requests runtime permission if needed. Then invoke the initUI() methods to initialize the UI elements.
Next, override the onRequestPermissionsResult() method to check the result of runtime permission request. And then invoke the startSDKRegistration() method to register the application.
Furthermore, implement the startSDKRegistration() method and invoke the registerApp() method of DJISDKManager to register the application. If the registration is successful, invoke the startConnectionToProduct() method of DJISDKManager inside the onRegister() callback method to start the connection between SDK and the DJI Products.
Once we finish the steps above, continue to add the code at the bottom of the onCreate() method:
// Register the broadcast receiver for receiving the device connection's changes. IntentFilter filter = new IntentFilter(); filter.addAction(FPVDemoApplication.FLAG_CONNECTION_CHANGE); registerReceiver(mReceiver, filter);
Here, we register the broadcast receiver for receiving the device connection's changes.
Next, add the following methods below the initUI() method:
protected BroadcastReceiver mReceiver = new BroadcastReceiver() {
In the code above, we implement the following features:
Create the "BroadcastReceiver" and override its onReceive() method to invoke the refreshSDKRelativeUI() method to refresh the UI elements.
We override the onDestroy() method and invoke the unregisterReceiver() method by passing the mReceiver variable to unregister the broadcast receiver.
In the refreshSDKRelativeUI() method, we check the BaseProduct's connection status by invoking isConnected() method. If the product is connected, we enable the mBtnOpen button, update the mTextConnectionStatus's text content and update the mTextProduct's content with product name. Otherwise, if the product is disconnected, we disable the mBtnOpen button and update the mTextProduct and mTextConnectionStatus textViews' content.
Finally, let's implement the onClick() method of mBtnOpen button and the showToast() method as shown below:
Here, we create an Intent object with the class of MainActivity and invoke the startActivity() method by passing intent object to start the MainActivity.
Implementing the First Person View
Now, let's open the "MainActivity.java" file and declare the TAG and mReceivedVideoDataListener variables as shown below:
privatestaticfinal String TAG = MainActivity.class.getName(); protected VideoFeeder.VideoDataListener mReceivedVideoDataListener = null;
// The callback for receiving the raw H264 video data for camera live view mReceivedVideoDataListener = new VideoFeeder.VideoDataListener() {
@Override publicvoidonReceive(byte[] videoBuffer, int size){ if (mCodecManager != null) { mCodecManager.sendDataToDecoder(videoBuffer, size); } } }; }
In the code above, we initialize the mReceivedVideoDataListener variable using VideoFeeder's VideoDataListener(). Inside the callback, we override its onReceive() method to get the raw H264 video data and send them to mCodecManager for decoding.
Next, let's implement the onProductChange() method invoke it in the onResume() method as shown below:
if (product == null || !product.isConnected()) { showToast(getString(R.string.disconnected)); } else { if (null != mVideoSurface) { mVideoSurface.setSurfaceTextureListener(this); } if (!product.getModel().equals(Model.UNKNOWN_AIRCRAFT)) { VideoFeeder.getInstance().getPrimaryVideoFeed().addVideoDataListener(mReceivedVideoDataListener); } } }
privatevoiduninitPreviewer(){ Camera camera = FPVDemoApplication.getCameraInstance(); if (camera != null){ // Reset the callback VideoFeeder.getInstance().getPrimaryVideoFeed().addVideoDataListener(null); } }
In the initPreviewer() method, firstly, we check the product connection status and invoke the setSurfaceTextureListener() method of TextureView to set texture listener to MainActivity. Then check if VideoFeeder has video feeds and the video feed's size is larger than 0 and set the mReceivedVideoDataListener as its "callback". So once the camera is connected and receive video data, it will show on the mVideoSurface TextureView.
Moreover, we implement the uninitPreviewer() method to reset Camera's "VideoDataListener" to null.
Now, let's override the four SurfaceTextureListener interface methods as shown below:
@Override publicvoidonSurfaceTextureAvailable(SurfaceTexture surface, int width, int height){ Log.e(TAG, "onSurfaceTextureAvailable"); if (mCodecManager == null) { mCodecManager = new DJICodecManager(this, surface, width, height); } }
@Override publicvoidonSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height){ Log.e(TAG, "onSurfaceTextureSizeChanged"); }
We init the mCodecManager variable in the onSurfaceTextureAvailable() method, then reset the mCodecManager and invoke its cleanSurface() method to reset the surface data.
For more detail implementations, please check the Github source code of this tutorial.
Connecting to the Aircraft or Handheld Device
After you finish the steps above, please check this Connect Mobile Device and Run Application guide to run the application and view the live video stream from your DJI product's camera based on what we've finished of the application so far!
Enjoying the First Person View
If you can see the live video stream in the application, congratulations! Let's move forward.
Implementing the Capture function
Now, let's override the onClick() method to implement the capture button click action:
Then declare a handler variable and initialize it in the onCreate() method as shown below:
private Handler handler;
handler = new Handler();
Next, implement the captureAction() method as shown below:
// Method for taking photo private void captureAction(){
final Camera camera = FPVDemoApplication.getCameraInstance(); if (camera != null) {
SettingsDefinitions.ShootPhotoMode photoMode = SettingsDefinitions.ShootPhotoMode.SINGLE; // Set the camera capture mode as Single mode camera.setShootPhotoMode(photoMode, new CommonCallbacks.CompletionCallback(){ @Override public void onResult(DJIError djiError) { if (null == djiError) { handler.postDelayed(new Runnable() { @Override public void run() { camera.startShootPhoto(new CommonCallbacks.CompletionCallback() { @Override public void onResult(DJIError djiError) { if (djiError == null) { showToast("take photo: success"); } else { showToast(djiError.getDescription()); } } }); } }, 2000); } } }); } }
In the code above, firstly, we create a "ShootPhotoMode" variable and assign "ShootPhotoMode.SINGLE" to it. Then invoke the setShootPhotoMode() method of Camera to set the shoot photo mode. The camera shoot photo mode has several modes within its definition. You can use "AEB", "BURST", "HDR", etc for "ShootPhotoMode", for more details, please check SettingsDefinitions.ShootPhotoMode.
Next, implement the startShootPhoto() method of Camera inside the completion callback of setShootPhotoMode method to control the camera to shoot photo. Here, we invoke the postDelayed() method of Handler to delay the method execution for 2000 milliseconds since the camera need time to execute the setShootPhotoMode command.
Finally, we override its onResult() method of startShootPhoto() to get the result and show the related texts to users.
Build and run your project and then try the shoot photo function. If the screen flashes after your press the Capture button, your capture function should work now.
Implementing the Record function
Switching Camera Mode
Before we go ahead to implement the record action method, let's implement the switch Camera Mode feature. Improve the onClick() method by adding button click actions for mShootPhotoModeBtn and mRecordVideoModeBtn as follows:
@Override publicvoidonClick(View v){
switch (v.getId()) { case R.id.btn_capture:{ captureAction(); break; } case R.id.btn_shoot_photo_mode:{ switchCameraMode(SettingsDefinitions.CameraMode.SHOOT_PHOTO); break; } case R.id.btn_record_video_mode:{ switchCameraMode(SettingsDefinitions.CameraMode.RECORD_VIDEO); break; } default: break; } }
Camera camera = FPVDemoApplication.getCameraInstance(); if (camera != null) { camera.setMode(cameraMode, new CommonCallbacks.CompletionCallback() { @Override publicvoidonResult(DJIError error){
if (error == null) { showToast("Switch Camera Mode Succeeded"); } else { showToast(error.getDescription()); } } }); } }
In the code above, we invoke the setMode() method of Camera and assign the cameraMode parameter to it. Then override the onResult() method to show the change camera mode result to the users.
Working on the Record Action
Once we finish the switch camera mode feature, we can now implement the record feature. Let's improve the initUI() method by add the following code at the bottom of it:
Here, we implement the setOnCheckedChangeListener() method of ToggleButton mRecordBtn and override its onCheckedChanged() method to check the isChecked variable value, which means the toggle state of the button, and invoke the startRecord() and stopRecord() methods relatively.
Next, implement the startRecord() and stopRecord() methods as shown below:
// Method for starting recording privatevoidstartRecord(){
final Camera camera = FPVDemoApplication.getCameraInstance(); if (camera != null) { camera.startRecordVideo(new CommonCallbacks.CompletionCallback(){ @Override publicvoidonResult(DJIError djiError) { if (djiError == null) { showToast("Record video: success"); }else { showToast(djiError.getDescription()); } } }); // Execute the startRecordVideo API } }
// Method for stopping recording privatevoidstopRecord(){
Camera camera = FPVDemoApplication.getCameraInstance(); if (camera != null) { camera.stopRecordVideo(new CommonCallbacks.CompletionCallback(){
In the code above, we invoke the startRecordVideo() and stopRecordVideo() methods of Camera to implement the start record and stop record features. And show the result messages to our user by overriding the onResult() methods.
Lastly, when the video starts recording, we should show the recording time info to our users. So let's add the following code to the bottom of onCreate() method as follows:
Camera camera = FPVDemoApplication.getCameraInstance();
/* * Update recordingTime TextView visibility and mRecordBtn's check state */ if (isVideoRecording){ recordingTime.setVisibility(View.VISIBLE); }else { recordingTime.setVisibility(View.INVISIBLE); } } }); } } });
}
Here, we implement the setSystemStateCallback() of Camera and override the onUpdate() method to get the current camera system state, we can call the getCurrentVideoRecordingTimeInSeconds() method of "SystemState" to get the record time info. Before we show the record time info to our users, we should convert it from seconds to "00:00" format including minutes and seconds. Lastly, we update the TextView recordingTime variable's text value with the latest record time info and update the visibility of recordingTime TextView in UI Thread.
For more details, please check the Github source code of this tutorial.
Now, let's build and run the project and check the functions. Here we use Mavic Pro as an example. You can try to play with the Capture, Record and Switch Camera WorkMode functions, here is a gif animation to demo these three functions:
Congratulations! Your Aerial FPV android app is complete, you can now use this app to control the camera of your DJI Product now.
Summary
In this tutorial, you’ve learned how to use DJI Mobile SDK to show the FPV View from the aircraft's camera and control the camera of DJI's Aircraft to shoot photo and record video. These are the most basic and common features in a typical drone mobile app: Capture and Record. However, if you want to create a drone app which is more fancy, you still have a long way to go. More advanced features should be implemented, including previewing the photo and video in the SD Card, showing the OSD data of the aircraft and so on. Hope you enjoy this tutorial, and stay tuned for our next one!