Overview

Using the video stream transmission control function provided by PSDK, the developer needs to realize the function of obtaining the code stream of the video stream file first, according to the H.264 encoding format encode the video stream, and call the specified interface to send the video stream data in combination with the video frame rate and other parameters. Users can obtain the real-time video data on the camera class payload device by using DJI Pilot and the Mobile App developed based on MSDK.

Related References

• Please refer to Video Standard for the detailed description of the video stream format.
• Please refer to H.264 code for related reference to H.264 standard code stream Streaming Standards.
• The video streaming function is only supported by the camera class payload device developed based on Linux.

Using the video streaming function

1. Configure network parameters

The developer can choose to set the network parameters of the load device automatically or manually according to the actual situation:

• Set network parameters in an automatic way

  In order to facilitate developers to use development platforms with network ports to achieve wider applications, PSDK recommends developers to use the DjiPlatform_RegHalNetworkHandler() interface to register the callback function for setting the network parameters of the load device, and automatically set the network parameters. Otherwise, the load device can only set the network parameters of the load device manually.

• Manually set network parameters

  If the DjiPlatform_RegHalNetworkHandler() interface is not called to register the callback function of the network configuration, the developer needs to manually configure the network parameters. The user can only use the video streaming function, and other functions involving the use of the network port will not be available.

Set network parameters in an automatic way (recommended)

When automatically setting the network parameters, the developer must use the DjiPlatform_RegHalNetworkHandler() interface to register the callback function for setting the network parameters of the load device.

• Construct the callback function

  Construct a callback function that sets network parameters

T_DjiReturnCode HalNetWork_Init(const char *ipAddr, const char *netMask, T_DjiNetworkHandle *halObj)
{
int32_t ret;
char cmdStr[LINUX_CMD_STR_MAX_SIZE];

if (ipAddr == NULL || netMask == NULL) {
USER_LOG_ERROR("hal network config param error");
return DJI_ERROR_SYSTEM_MODULE_CODE_INVALID_PARAMETER;
}

//Attention: need root permission to config ip addr and netmask.
memset(cmdStr, 0, sizeof(cmdStr));

snprintf(cmdStr, sizeof(cmdStr), "ifconfig %s up", LINUX_NETWORK_DEV);
ret = system(cmdStr);
if (ret != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
USER_LOG_ERROR("Can't open the network."
"Probably the program not execute with root permission."
"Please use the root permission to execute the program.");
return DJI_ERROR_SYSTEM_MODULE_CODE_SYSTEM_ERROR;
}

snprintf(cmdStr, sizeof(cmdStr), "ifconfig %s %s netmask %s", LINUX_NETWORK_DEV, ipAddr, netMask);
ret = system(cmdStr);
if (ret != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
USER_LOG_ERROR("Can't config the ip address of network."
"Probably the program not execute with root permission."
"Please use the root permission to execute the program.");
return DJI_ERROR_SYSTEM_MODULE_CODE_SYSTEM_ERROR;
}

return DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS;
}

• Register callback function

  After registering the DjiPlatform_RegHalNetworkHandler() callback function, the user uses the sudo command (administrator privilege) to execute the program of the payload device, and realizes the automatic configuration of the network parameters of the payload device by obtaining the network configuration information negotiated between the drone and the payload device.

Note: For Linux systems with the network-manager service, you need to delete all the configuration information of the network-manager service, otherwise the network parameters cannot be successfully configured.

returnCode = DjiPlatform_RegHalNetworkHandler(&networkHandler);
if (returnCode != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
printf("register hal network handler error");
return DJI_ERROR_SYSTEM_MODULE_CODE_SYSTEM_ERROR;
}

Set network parameters manually

When setting the network parameters of the camera-like load device by manually, the user** can only use the video streaming function, and other functions involving the use of the network port will not be available**.

• IP address: 192.168.5.3
• Gateway: 192.168.5.1
• Subnet mask: 255.255.255.0 After the IP address is set, use the ping and ifconfig commands to view the network status between the camera-like payload and the aircraft.

2. Create a video stream processing thread

• Create a video stream processing thread

  In order to avoid blocking the video stream processing thread due to other tasks, resulting in the problem of blurred screen and green screen during video stream transmission, please create a video stream processing thread separately when using PSDK to develop a camera-like payload device.

returnCode = osalHandler->TaskCreate("user_camera_media_task", UserCameraMedia_SendVideoTask, 2048, NULL, &s_userSendVideoThread);
if (returnCode != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
USER_LOG_ERROR("user send video task create error.");
return DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS;
}

• Video stream processing thread initialization

  After creating a video stream processing thread, a camera-like payload device developed using PSDK needs to initialize the thread state and apply to the camera-like payload device for memory space for caching video stream files.

videoFilePath = osalHandler->Malloc(DJI_FILE_PATH_SIZE_MAX);
if (videoFilePath == NULL) {
USER_LOG_ERROR("malloc memory for video file path fail.");
exit(1);
}

transcodedFilePath = osalHandler->Malloc(DJI_FILE_PATH_SIZE_MAX);
if (transcodedFilePath == NULL) {
USER_LOG_ERROR("malloc memory for transcoded file path fail.");
exit(1);
}

frameInfo = osalHandler->Malloc(VIDEO_FRAME_MAX_COUNT * sizeof(T_TestPayloadCameraVideoFrameInfo));
if (frameInfo == NULL) {
USER_LOG_ERROR("malloc memory for frame info fail.");
exit(1);
}
memset(frameInfo, 0, VIDEO_FRAME_MAX_COUNT * sizeof(T_TestPayloadCameraVideoFrameInfo));

Compatibility Notes

• PSDK V2.0.0 and above supports interfaces for video stream sending and video stream status information acquisition. Developers need to call the video stream sending interface according to the video stream status (bit rate and other information) to send a video stream that conforms to the encoding specification.
• The sample program of PSDK V1.5.x video stream can be used with the program of PSDK V2.0.0 and above. When using it, please do not register the callback function halNetWorkHandler(), and manually set the IP of the load device. Otherwise the example will not work.

3. Get video stream file information

Before sending a video stream file, a camera payload device developed with PSDK must read the local H.264 file of the camera payload device to obtain the information of the video stream file.

• Read the local H.264 file of the camera class payload

  After the user specifies the exact path of the H.264 file on the payload device, the camera-like payload device developed based on PSDK opens the video stream file specified by the user through the system interface.

Note: The video streaming function of PSDK only supports the transmission of video streaming files in H.264 format. For details of the H.264 format standard, please refer to "Video Standard".

returnCode = DjiUserUtil_GetCurrentFileDirPath(__FILE__, DJI_FILE_PATH_SIZE_MAX, curFileDirPath);
if (returnCode != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
USER_LOG_ERROR("Get file current path error, stat = 0x%08llX", returnCode);
exit(1);
}
snprintf(tempPath, DJI_FILE_PATH_SIZE_MAX, "%smedia_file/PSDK_0005.h264", curFileDirPath);

videoFilePath = osalHandler->Malloc(DJI_FILE_PATH_SIZE_MAX);
if (videoFilePath == NULL) {
USER_LOG_ERROR("malloc memory for video file path fail.");
exit(1);
}

fpFile = fopen(transcodedFilePath, "rb+");
if (fpFile == NULL) {
USER_LOG_ERROR("open video file fail.");
continue;
}

• Get H.264 file information

  The camera class load device developed based on PSDK uses FFmpeg to read the frame rate, frame information and total number of frames of the specified H.264 file.

  • Frame rate: The number of frames that the camera-like payload device can send in 1s
  • Frame information: the starting position of a frame in the H.264 video stream file and the length of the frame

// video send preprocess
returnCode = DjiPlayback_VideoFileTranscode(videoFilePath, "h264", transcodedFilePath,
DJI_FILE_PATH_SIZE_MAX);
if (returnCode != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
USER_LOG_ERROR("transcode video file error: 0x%08llX.", returnCode);
continue;
}

returnCode = DjiPlayback_GetFrameRateOfVideoFile(transcodedFilePath, &frameRate);
if (returnCode != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
USER_LOG_ERROR("get frame rate of video error: 0x%08llX.", returnCode);
continue;
}

returnCode = DjiPlayback_GetFrameInfoOfVideoFile(transcodedFilePath, frameInfo, 	VIDEO_FRAME_MAX_COUNT, &frameCount);
if (returnCode != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
USER_LOG_ERROR("get frame info of video error: 0x%08llX.", returnCode);
continue;
}

returnCode = DjiPlayback_GetFrameNumberByTime(frameInfo, frameCount, &frameNumber,
startTimeMs);
if (returnCode != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
USER_LOG_ERROR("get start frame number error: 0x%08llX.", returnCode);
continue;
}

if (fpFile != NULL)
fclose(fpFile);

4. Video stream file parsing

After the camera-like payload device developed based on PSDK obtains the information of the video stream file, it will parse the content of the video stream file and identify the frame header of the video stream file.

dataBuffer = calloc(frameBufSize, 1);
if (dataBuffer == NULL) {
USER_LOG_ERROR("malloc fail.");
goto free;
}

ret = fseek(fpFile, frameInfo[frameNumber].positionInFile, SEEK_SET);
if (ret != 0) {
USER_LOG_ERROR("fseek fail.");
goto free;
}

dataLength = fread(dataBuffer, 1, frameInfo[frameNumber].size, fpFile);
if (dataLength != frameInfo[frameNumber].size) {
USER_LOG_ERROR("read data from video file error.");
} else {
USER_LOG_DEBUG("read data from video file success, len = %d B\r\n", dataLength);
}

5. Send video stream file

The camera payload device developed based on PSDK will send the video stream file frame by frame after parsing the video stream file and identifying the frame header of the video stream file.

Note: The longest frame that can be sent by a camera payload device developed based on PSDK is 65K. If the length of the frame exceeds 65K, the frame needs to be disassembled and sent.

lengthOfDataHaveBeenSent = 0;
while (dataLength - lengthOfDataHaveBeenSent) {
lengthOfDataToBeSent = USER_UTIL_MIN(DATA_SEND_FROM_VIDEO_STREAM_MAX_LEN,
dataLength - lengthOfDataHaveBeenSent);
returnCode = DjiPayloadCamera_SendVideoStream((const uint8_t *) dataBuffer + lengthOfDataHaveBeenSent,
lengthOfDataToBeSent);
if (returnCode != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
USER_LOG_ERROR("send video stream error: 0x%08llX.", returnCode);
}
lengthOfDataHaveBeenSent += lengthOfDataToBeSent;
}

After using DJI Pilot or a Mobile App developed based on MSDK to send a video streaming command to the camera-like payload, the Mobile App will be able to receive and loop the media files in the camera-like payload.

6. Adjust the frame rate

The camera-like payload device developed with PSDK can update the status of video stream transmission, which is convenient for users to adjust the frame rate of the video transmitted by the video stream transmission module.

returnCode = DjiPayloadCamera_GetVideoStreamState(&videoStreamState);
if (returnCode == DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
USER_LOG_DEBUG(
"video stream state: realtimeBandwidthLimit: %d, realtimeBandwidthBeforeFlowController: %d, realtimeBandwidthAfterFlowController:%d busyState: %d.",
videoStreamState.realtimeBandwidthLimit, videoStreamState.realtimeBandwidthBeforeFlowController,
videoStreamState.realtimeBandwidthAfterFlowController,
videoStreamState.busyState);
} else {
USER_LOG_ERROR("get video stream state error.");
}