If you come across any mistakes or bugs in this tutorial, please let us know using a Github issue, a post on the DJI Forum. Please feel free to send us Github pull request and help us fix any issues.
In this tutorial, you will learn how to use the DJISimulator in your Android Studio project using DJI Mobile SDK. With the help of Virtual Stick control, you can input Virtual Stick flight control data and check the changes of simulator state in real time.
You can download the tutorial's final sample project from this Github Page.
We use Mavic Pro as an example to make this demo.
Let's get started!
DJISimulator is used to control the aircraft in a simulated environment based on the virtual stick input. The simulated aircraft state information will also be displayed on the screen.
You can use the
Simulator class in
FlightController to control the simulation. It allows both manual and automated flights to be simulated without actually flying the aircraft.
Additionally, simulator initialization, monitoring and termination can be controlled directly through the SDK allowing for application development in continuous integration environments.
For DJI SDK mobile application used in China, it's required to activate the application and bind the aircraft to the user's DJI account.
If an application is not activated, the aircraft not bound (if required), or a legacy version of the SDK (< 4.1) is being used, all camera live streams will be disabled, and flight will be limited to a zone of 100m diameter and 30m height to ensure the aircraft stays within line of sight.
To learn how to implement this feature, please check this tutorial Application Activation and Aircraft Binding.
In the Importing and Activating DJI SDK in Android Studio Project tutorial, you have learned how to import the DJI Android SDK into your Android Studio project and activate your application. If you haven't read that previously, please take a look at it. Once you've done that, let's continue to create the project.
1. Open Android Studio and select File -> New -> New Project to create a new project, named 'DJISimulatorDemo'. Enter the company domain and package name (Here we use "com.dji.simulatorDemo") you want and press Next. Set the minimum SDK version as
API 19: Android 4.4 (KitKat) for "Phone and Tablet" and press Next. Then select "Empty Activity" and press Next. Lastly, leave the Activity Name as "MainActivity", and the Layout Name as "activity_main", press "Finish" to create the project.
2. Unzip the Android SDK package downloaded from DJI Developer Website. Go to File -> New -> Import Module, enter the "API Library" folder location of the downloaded Android SDK package in the "Source directory" field. A "dJISDKLIB" name will show in the "Module name" field. Press Next and Finish button to finish the settings.
3. Next, double click on the "build.gradle(Module: app)" in the project navigator to open it and replace the content with the followings:
Here, we modify its dependencies by adding
compile project(':dJISDKLIB') in the "dependencies" part at the bottom, and change the compileSdkVersion, buildToolsVersion number, etc.
Then, select the Tools -> Android -> Sync Project with Gradle Files on the top bar and wait for Gradle project sync finish.
4. Let's right click on the 'app' module in the project navigator and click "Open Module Settings" to open the Project Struture window. Navigate to the "Dependencies" tab, you should find the "dJISDKLIB" appear in the list. Your SDK environmental setup should be ready now!
5. Now, open the MainActivity.java file in
com.dji.simulatorDemo package and add
import dji.sdk.sdkmanager.DJISDKManager; at the bottom of the import classes section as shown below:
Wait for a few seconds and check if the words turn red, if they remain gray color, it means you can use DJI Android SDK in your project successfully now.
Right-click on the package
com.dji.simulatorDemo in the project navigator and choose New -> Java Class, Type in "DJISimulatorApplication" in the Name field and select "Class" as Kind field content.
Next, Replace the code of the "DJISimulatorApplication.java" file with the following:
Here, we override the onCreate() method. We can do some settings when the application is created here.
In order to input some simulated data, like
verticalThrottle, you may need a joystick control. Let's work on the implementation of it.
We implement the joystick control base on an open source Github project OnScreenJoystick . You can download the Github project to get the OnScreenJoystick.java and OnScreenJoystickListener.java files or get them from this tutorial's Github Sample project. Now, copy and paste these two java files to the folder of "com.dji.simulatorDemo" on the left navigator as shown below:
When you touch on the OnScreenJoystick view, the
onTouch method in OnScreenJoystickListener will be called and you can get the "OnScreenJoystick" joystick object, and the x and y coordinate of the knob parameters from this method as shown below:
Note: The values for x coordinate of the knob are between -1 (left) and 1 (right), the values for y coordinate of the knob are between -1 (down) and 1 (up).
Next, copy and paste the joystick.png and joystick_bg.png files from this tutorial's Github sample project to the mipmap folder on the left navigator as shown below:
Now, let's open the MainActivity.java file and replace the code with the followings:
In the code above, we implement the following features:
1. Create the layout UI elements variables, including two TextView
mTextView, four buttons
mBtnLand, one Toggle Button
mBtnSimulator and two OnScreenJoystick control
2. In the
onCreate() method, we request several permissions at runtime to ensure the SDK works well when the compile and target SDK version is higher than 22(Like Android Marshmallow 6.0 device and API 23). Then invoke the
initUI() method to initialize UI variables and their listeners.
3. In the
initUI() method, we first initialize the UI elements variables, then set the click listener of the four buttons to "this". Moreover, implement the
onCheckedChanged() method of toggle button
mBtnSimulator's "setOnCheckedChangeListener". Finally, implement the
onTouch() method of the two
OnScreenJoystick objects' "setJoystickListener".
4. Override the
onClick() method to implement the four buttons' click actions.
Open the activity_main.xml layout file and replace the code with the following:
In the xml file, first, we implement the RelativeLayout element. We declare an ImageButton(id: ReturnBtnCamera) element to exit the application, and a TextView(id: ConnectStatusTextView) element to show the connection status text.
Next, create a TextureView(id: textview_simulator) element to show the simulator state infos. Moreover, create the "Enable Virtual Stick" button(id: btn_enable_virtual_stick), "Disable Virtual Stick" button(id: btn_disable_virtual_stick), "Take Off" button(id: btn_take_off), "Land" button(id: btn_land). Moreover, create the "Start Simulator" toggle button(id: btn_start_simulator) and set its textOn and textOff params to "Start Simulator" and "Stop Simulator".
Lastly, we create two OnScreenJoystick elements (id: directionJoystickRight) and (id:directionJoystickLeft) for joystick control.
Once you finish the above steps, let's add some resources files to the res folder on the left navigator of Android Studio.
Copy the following image and xml files from the tutorial Github Sample project's drawable folder to your project, they are used for the button's UI:
Next, open the "colors.xml" file and add the following code at the bottom to declare the black overlay color:
Moreover, open the "strings.xml" file and add the "success" string:
Lastly, open the "styles.xml" file and add the following code to declare the "common_button" style:
Once you finish the above steps, let's register the application on DJI Developer Website and get the App Key. If you are not familiar with the App Key, please check Generate an App Key for details.
1. Open the AndroidManifest.xml file and add the following elements on top of the application element:
Here, we request permissions that the application must be granted in order for it to register DJI SDK correctly. Also we declare the camera and usb hardwares which is used by the application.
Moreover, let's add the following elements as childs of element on top of the "MainActivity" activity element as shown below:
In the code above, you should substitude your App Key of the application for "Please enter your App Key here." in the value attribute under the
2. After you finish the steps above, open the "DJISimulatorApplication.java" file and replace the code with the same file in the Github Source Code, here we explain the important parts of it:
Here, we implement several features:
onCreate()method to initialize the DJISDKManager.
SDKManagerCallback. You can use the
onRegister()method to check the Application registration status and show text message here. Using the
onProductChange()method, we can check the product connection status and invoke the
notifyStatusChange()method to notify status changes.
BaseProductListener. You can use the
onComponentChange()method to check the product component change status and invoke the
notifyStatusChange()method to notify status changes. Also, you can use the
onConnectivityChange()method to notify the product connectivity changes.
Now let's build and run the project and install it to your Android device. If everything goes well, you should see the "success" textView like the following screenshot when you register the app successfully.
Important: Please check if the "armeabi-v7a", "arm64-v8a" and "x86" lib folders has been added to your jnLibs folder in dJISDKLib successfully before testing resgistering the app.
For more details of integrating and activating the SDK in Android Studio, please check: Android Studio Project Integration.
Now, let's update the product connection status. Open the "MainActivity.java" file and add the following code at the bottom of
The code above register the broadcast receiver for receiving the device connection's changes.
Next, add the following four methods below
onCreate() and override the
onDestroy() methods as shown below:
As the code shown above, we implement the following features:
1. Create a BroadcastReceiver object
mReceiver, override its
onReceive() method and invoke the
updateTitleBar() method to update the
2. Create the
showToast() method to display the toast notification message to users.
3. In the
updateTitleBar() method, we first check if mConnectStatusTextView is null, then create a BaseProduct object by invoking the
getProductInstance() method of DJISimulatorApplication.
Moreover, invoke the
isConnected() method of BaseProduct to check if the product is connected, then invoke the
getModel() method of BaseProduct to get the model name and show it in
mConnectStatusTextView. If the product is not connected, cast the
product object as Aircraft object, and check if the remoteController is not null and if it's connected, then update the
mConnectStatusTextView's text content.
Lastly, if the product or remote controller are not connected, then update the
mConnectStatusTextView's text with "Disconnected".
4. We override the
onResume() method to invoke the
updateTitleBar() method to update
mConnectStatusTextView when the activity start interacting with the user. Then override the
onDestroy() method to unregister the BroadcastReceiver object.
Now let's build and run the project and install it to your Android device. Then connect the demo application to your Mavic Pro (Please check Run Application for more details), if everything goes well, you should see the title textView content updates to "MavicPro Connected" as shown below:
Since we have implemented the Joystick control, now let's continue to work on sending virtual stick flight control data to the aircraft. First, we create a DJIFlightController variable
mFlightController, a Timer variable
mSendVirtualStickDataTimer, a SendVirtualStickDataTask(extends from TimerTask class) variable
mSendVirtualStickDataTask and four float variables on top of
onCreate() method as shown below:
We may use the
mThrottle variables to store the pitch, roll, yaw and vertical throttle virtual stick flight control data.
Next, create the
initFlightController() method, invoke it in the
onResume() method and implement the SendVirtualStickDataTask class as shown below:
In the code above, we implement the following features:
1. In the
initFlightController() method, we first check if the aircraft is not null and is connected, then invoke the
getFlightController() method of Aircraft to get the
2. Next, extends from TimerTask class to create the SendVirtualStickDataTask class. Inside the class, override the
run() method to invoke the
sendVirtualStickFlightControlData() method of FlightController to send virtual stick flight control data. Here, we create the FlightControlData object from the four float variables declared before:
Once you finish the above steps, let's implement the
setJoystickListener() methods of
mScreenJoystickRight variables at the bottom of
initUI() method as shown below:
Here, we implement the following features:
1. Override the
onTouch() method of setJoystickListener and filter the
pY variables' value by checking if they are less than 0.02. We should not send the virtual stick data to flight controller too frequently if the value is too small.
2. Get the maximum velocity of pitch and roll control, then store them to
rollJoyControlMaxSpeed variables. Since the value of
pX is between -1 (left) and 1 (right), the value of
pY is between -1 (down) and 1 (up), we multiply by using the
rollJoyControlMaxSpeed values to update the
mRoll data. Here we take Mode 2(American mode) of remote controller as example.
3. Lastly, we check if
mSendVirtualStickDataTimer is null, and create it by invoking the
SendVirtualStickDataTask() method. Then, create the
mSendVirtualStickDataTimer and invoke its
schedule() method to trigger the timer by passing
mSendVirtualStickDataTask variable, 0 milliseconds of delay and 200 milliseconds between subsequent executions.
4. Similarly, implement the
setJoystickListener() method of
mScreenJoystickRight variable to update the
mThrottle values and trigger the timer to send virtual stick data to the aircraft's flight controller.
Now, when you control the left and right joysticks, they will send the simulated virtual stick data (Including Yaw, Pitch, Roll and Vertical Throttle) to the flight controller of aircraft.
Lastly, override the
onClick() method to implement the enable and disable virtual stick control buttons click actions as shown below:
This invoke the
disableVirtualStickControlMode() methods of FlightController to enable and disable the virtual stick control mode.
Let's implement the DJISimulator feature now. In order to update the simulator state data in
mTextView, we may need to implement the
setStateCallback() method of DJISimulator in the
initFlightController() method as shown below:
In the code above, we override the
onUpdate() method to get the lastest simulator state data, then invoke the
getPositionZ() methods of
SimulatorState to get the updated yaw, pitch, roll, positionX, positionY and positionZ values and show them in
Next, override the
onCheckedChanged() method of
setOnCheckedChangeListener() method as shown below:
In the code above, we implement the following features:
1. If the
mBtnSimulator toggle button is checked, then show the
mTextView. Next, if the
mFlightController is not null, we invoke the
start() method of DJISimulator by passing a
InitializationData with LocationCoordinate2D struct (lattitude 23 and longitude 113), updateFrequency 10 and satelliteCount 10 parameters to it. For more details of DJISimulatorInitializationData, please check the Android API Reference.
2. Next, overide the
onResult() method of
showToast() method to show the start simulator result to the user.
3. Similarly, if the
mBtnSimulator toggle button is not checked, then invoke the
stop() method of DJISimulator to stop the simulator. Furthermore, override the
onResult() method and invoke the
showToast() method to show the stop simulator result to the user.
Finally, let's add the following code at the bottom of
onClick() method to implement the Take off and Land buttons' click actions as shown below:
For the case of "R.id.btn_take_off", we invoke the
startTakeoff() method of FlightController to send the take off command to the aircraft. Similiarly, for the case of "R.id.btn_land", we invoke the
startLanding() method to send the auto landing command. It's just that simple and easy.
We have gone through a long way in this tutorial, now let's build and run the project, connect the demo application to your Mavic Pro (Please check Run Application for more details) and check all the features we have implemented so far.
If everything goes well, you should see something similiar to the following gif animations like this:
If the demo application is connected with Mavic Pro successfully, you should see the title textView content updates to "MavicPro Connected".
Press Enable Virtual Stick button to enable virtual stick control, then press Start Simulator to start the simulator.
Moreover, press the Take Off button to send take off command to the aircraft, if the command executes successfully, you should see the PosZ value start to change, means that the aircraft is rising.
Now you can drag the left and right virtual stick controls to simulate the flight behavious.
Lastly, press the Land button to make the aircraft start auto landing, once it finish, you may notice the PosZ value becomes "0.00". Press the Stop Simulator button to stop the simulator and then press Disable Virtual Stick to disable the virtual stick control.
In this tutorial, you've learned how to use the DJISimulator feature to simulate aircraft's flight behaviour in a simulated environment based on the virtual stick control input and show the changes of simulator state(Yaw,Pitch,Roll,PosX,PosY and PosZ) in real time. Also you've learned how to use Virtual Stick control to send virtual stick flight control data to the aircraft.
This demo is a simple demonstration of using DJISimulator, to have a better user experience, you can create a 3D simulated environment using 3D game engine like Unity3D to show the simulated data and aircraft flight behavious inside your mobile application (Like the Flight Simulator in DJI Go app)!
Furthermore, the DJISimulator allows for automated testing in continous integration environment(Like Jenkins), it would help your DJI-SDK based application testing process. Good luck, and hope you enjoyed this tutorial!