Difference between revisions of "F16: Micro Watch Monitoring System"

From Embedded Systems Learning Academy
Jump to: navigation, search
(Created page with "== Distributed Monitoring System with Alexa == == Abstract == The Distributed Monitoring System uses Amazon's Alexa Voice Service to relay monitored data of a network of SJ-O...")
 
(Tasks)
 
(95 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Distributed Monitoring System with Alexa ==
+
== Micro Watch Monitoring System ==
  
 
== Abstract ==
 
== Abstract ==
The Distributed Monitoring System uses Amazon's Alexa Voice Service to relay monitored data of a network of SJ-One Boards with the assistance of the Raspberry Pi 3. The monitored data from a SJ-One Board is transmitted and received via a Xbee Wire Antenna module. The monitored data is then uploaded to our backend system, Google Firebase, by the Raspberry Pi 3 and is accessible through any device integrated with the Alexa Voice Service.
+
The Micro Watch Monitoring System uses Amazon's Alexa Voice Service to relay monitored data from a network of SJ-One Boards with the assistance of the Raspberry Pi 3. The monitored data from a SJ-One Board is transmitted and received via a Xbee Wire Antenna module. The monitored data is then uploaded to our backend system, Google Firebase, by the Raspberry Pi 3, and is accessible through any device integrated with the Alexa Voice Service.
  
 
== Objectives & Introduction ==
 
== Objectives & Introduction ==
Show list of your objectives. This section includes the high level details of your project.  You can write about the various sensors or peripherals you used to get your project completed.
+
The objective of this project is to be able to monitor a network of SJ-One Boards for statistical data that is accessible via voice commands acknowledged by the Alexa Voice Service.
 +
 
 +
=== Tasks ===
 +
* Create FreeRTOS tasks to monitor sensor data, pin data, and OS statistics on the SJ-One Board
 +
* Create a task that parses sensor data into Javascript Object Notation (json) and utilizes the Xbee Wire Antenna to wirelessly transmit it via the UART protocol
 +
* Create a python script for the Raspberry Pi 3 that parses data received by the Xbee Wire Antenna and relays it to Firebase
 +
* Create an Alexa Skill that can answer user queries by accessing data in the Firebase Real-time Database.
  
 
=== Team Members & Responsibilities ===
 
=== Team Members & Responsibilities ===
 
[https://www.linkedin.com/in/jaywal Sanjay Agarwal]
 
[https://www.linkedin.com/in/jaywal Sanjay Agarwal]
* SJ-One Board Firmware Design
+
* SJ-One Board Software Design
 
* Hardware Connectivity Design
 
* Hardware Connectivity Design
* Firmware and Hardware System Testing
+
* Software and Hardware System Testing
  
 
[https://www.linkedin.com/in/masonitkin Mason Itkin]
 
[https://www.linkedin.com/in/masonitkin Mason Itkin]
* Raspberry Pi 3 Firmware Design
+
* Raspberry Pi 3 Software Design
 
* Amazon Alexa Voice Service Skill Design
 
* Amazon Alexa Voice Service Skill Design
* Firmware and Voice Service System Testing
+
* Software and Voice Service System Testing
  
 
== Schedule ==
 
== Schedule ==
Show a simple table or figures that show your scheduled as planned before you started working on the project.  Then in another table column, write down the actual schedule so that readers can see the planned vs. actual goals.  The point of the schedule is for readers to assess how to pace themselves if they are doing a similar project.
 
 
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
 
! scope="col"| Week
 
! scope="col"| Week
! scope="col"| Date
+
! scope="col"| Begin Task
! scope="col"| Task
 
 
! scope="col"| Task Description
 
! scope="col"| Task Description
 
! scope="col"| Result
 
! scope="col"| Result
 +
! scope="col"| Planned Completion
 +
! scope="col"| Actual Completion
 
|-
 
|-
 
! scope="row"| 1
 
! scope="row"| 1
| 10/21
+
| <center> 10/21 </center>
| Rough System Design
+
| <center> Group Research on System Design </center>
| - Raspberry Pi3 to run Amazon Alexa
+
| <center> Complete </center>
- SJSUOne Board used to control car motors
+
| <center> 10/27 </center>
|Complete
+
| <center> 10/27 </center>
 
|-
 
|-
 
! scope="row"| 2
 
! scope="row"| 2
| 10/28
+
| <center> 10/28 </center>
| Parts Order (pt. 1)
+
| <center> Design Group Task Structure </center>
| Raspberry Pi3 + microphone + SD Card (plans to order screen/case and car parts at a later date)
+
| <center> Complete </center>
|Complete
+
| <center> 11/3 </center>
 +
| <center> 11/3 </center>
 
|-
 
|-
 
! scope="row"| 3
 
! scope="row"| 3
| 11/4
+
| <center> 11/4 </center>
| Create group organizational structure
+
| <center> Order Raspberry Pi 3 </center>
| Start Github repo and Trello board
+
| <center> Complete </center>
|Complete
+
| <center> 11/10 </center>
 +
| <center> 11/10 </center>
 
|-
 
|-
 
! scope="row"| 4
 
! scope="row"| 4
| 11/11
+
| <center> 11/11 </center>
| Start Alexa Dev.
+
| <center> Load NOOBs OS onto Raspberry Pi 3 and Deploy Alexa Skills on Amazon Web Services </center>
| - NOOBs Loaded onto rPi3
+
| <center> Complete </center>
+
| <center> 11/17 </center>
- Skill deployment location created on AWS Lambda
+
| <center> 11/17 </center>
 
 
|Complete
 
 
|-
 
|-
 
! scope="row"| 5
 
! scope="row"| 5
| 11/18
+
| <center> 11/18 </center>
| Implement Alexa basic functionality
+
| <center> Implement Alexa Voice Interactions and Communication to Raspberry Pi 3 via Lambda Function </center>
| - All basic voice interactions
+
| <center> Complete </center>
 
+
| <center> 11/24 </center>
- Communication from lambda function to rPI3
+
| <center> 11/24 </center>
 
 
|Complete
 
 
|-
 
|-
 
! scope="row"| 6
 
! scope="row"| 6
| 11/25
+
| <center> 11/25 </center>
| Parts Order (pt. 2)
+
| <center> Order Xbee Wire Antenna Modules and Xbee Explorer Dongle </center>
| - Chassis/Mechanical Parts
+
| <center> Complete </center>
- DC Motors
+
| <center> 12/1 </center>
 
+
| <center> 12/1 </center>
- Accelerometer
 
 
 
- Hardware Sensors
 
|Incomplete
 
 
|-
 
|-
 
! scope="row"| 7
 
! scope="row"| 7
| 12/2
+
| <center> 12/2 </center>
| Start Robot Dev.
+
| <center> Design SJ-One Board Monitoring Task and Xbee Driver </center>
| - Build robot car
+
| <center> Complete </center>
- Ensure sensors work with SJSUOne Board
+
| <center> 12/8 </center>
|Incomplete
+
| <center> 12/8 </center>
 
|-
 
|-
 
! scope="row"| 8
 
! scope="row"| 8
| 12/9
+
| <center> 12/9 </center>
| Implement Alexa functionality with Robot
+
| <center> Design Raspberry Pi 3 Communication Protocol </center>
| Ensure that Alexa commands are able to work with the robot car
+
| <center> Complete </center>
|Incomplete
+
| <center> 12/15 </center>
 +
| <center> 12/15 </center>
 
|-
 
|-
 
! scope="row"| 9
 
! scope="row"| 9
| 12/16
+
| <center> 12/16 </center>
| Test the Alexa Assisted Robot
+
| <center> Test Firmware, Hardware, and Voice Service Interconnectivity </center>
| Ensure that the project's functionality passes all tests to prepare for presentation
+
| <center> Complete </center>
|Incomplete
+
| <center> 12/17 </center>
 +
| <center> 12/19 </center>
 
|-
 
|-
 
! scope="row"| 10
 
! scope="row"| 10
| 12/20
+
| <center> 12/21 </center>
| Project Presentation
+
| <center> Present Completed Project </center>
| Present our final version of the Alexa Assisted Robot to the class
+
| <center> Complete </center>
|Incomplete
+
| <center> 12/21 </center>
 +
| <center> 12/21 </center>
 
|}
 
|}
  
 
== Parts List & Cost ==
 
== Parts List & Cost ==
Give a simple list of the cost of your project broken down by components. Do not write long stories here.
+
{| class="wikitable"
 +
|-
 +
! scope="col"| Part
 +
! scope="col"| Quantity
 +
! scope="col"| Price
 +
! scope="col"| Merchant
 +
|-
 +
| <center> SJ-One Board </center>
 +
| <center> 1 </center>
 +
| <center> $80 </center>
 +
| <center> Preetpal Kang </center>
 +
|-
 +
| <center> Raspberry Pi 3 </center>
 +
| <center> 1 </center>
 +
| <center> $35.70 </center>
 +
| <center> Amazon </center>
 +
|-
 +
| <center> Raspberry Pi 3 NOOBs OS </center>
 +
| <center> 1 </center>
 +
| <center> $9.44 </center>
 +
| <center> Amazon </center>
 +
|-
 +
| <center> Xbee Wire Antenna </center>
 +
| <center> 2 </center>
 +
| <center> $24.95 </center>
 +
| <center> Amazon </center>
 +
|-
 +
| <center> Xbee Explorer Dongle </center>
 +
| <center> 1 </center>
 +
| <center> $26.95 </center>
 +
| <center> Amazon </center>
 +
|}
  
 
== Design & Implementation ==
 
== Design & Implementation ==
The design section can go over your hardware and software design. Organize this section using sub-sections that go over your design and implementation.
+
=== Hardware Interface ===
 +
 
 +
'''Micro Watch System Configuration'''
 +
 
 +
The Micro Watch system consists of the SJ-One Board, Raspberry Pi 3, Xbee Series 1 modules, and the Xbee Explorer Dongle.
 +
 
 +
 
 +
[[File:CMPE146_F16_MW_System.JPG | frame | center | <center> '''Figure 1. Micro Watch System Configuration''' </center>]]
 +
 
 +
 
 +
[[File:CMPE146_F16_MW_MicroWatch_System_Diagram.jpg | frame | center | <center> '''Figure 2. Micro Watch System Configuration''' </center>]]
 +
 
 +
 
 +
'''Xbee 1mW Wire Antenna - Series 1 '''
 +
 
 +
The Digi XCTU software is used to configure the Xbee Wire Antenna modules for wireless connection. A shared channel and Personal Area Network (PAN) are set for both of the modules as well as a destination and source address. Setting these variables is all that is needed to enable communication. The module that is placed on the SJ-One Board is setup as the coordinator while the module connected to the Raspberry Pi 3 via the Xbee Explorer Dongle is setup as an endpoint. Setting either modules would have the same effect in a Xbee network with two modules, but as this project is aimed at providing multiple SJ-One Boards with sensor reporting ability, this will be important later. The SJ-One Board sends data to the Raspberry Pi 3 via UART3 protocol, and the Raspberry Pi 3 parses this data upon receiving it.
 +
 
 +
 
 +
[[File:CMPE146_F16_MW_XbeeModule.jpg | frame | center | <center> '''Figure 3. Xbee Wire Antenna''' </center>]]
 +
 
 +
 
 +
'''Xbee Explorer Dongle'''
 +
 
 +
The Xbee Explorer Dongle is used to connect the Xbees to the Digi XCTU software as well as connect the Xbee to the Raspberry Pi 3. The Explorer dongle provides an interface between the Xbee pins and USB, allowing UART communication and configuration.
 +
 
 +
 
 +
[[File:CMPE146_F16_MW_ExplorerUSB.JPG | frame | center | <center> '''Figure 4. Xbee Explorer Dongle USB Controller''' </center>]]
  
=== Hardware Design ===
 
Discuss your hardware design here.  Show detailed schematics, and the interface here.
 
  
=== Hardware Interface ===
+
[[File:CMPE146_F16_MW_ExplorerPIN.JPG | frame | center | <center> '''Figure 5. Xbee Explorer Dongle Pin Configuration''' </center>]]
In this section, you can describe how your hardware communicates, such as which BUSes used. You can discuss your driver implementation here, such that the '''Software Design''' section is isolated to talk about high level workings rather than inner working of your project.
 
  
 
=== Software Design ===
 
=== Software Design ===
Show your software design. For example, if you are designing an MP3 Player, show the tasks that you are using, and what they are doing at a high level. Do not show the details of the code.  For example, do not show exact code, but you may show psuedocode and fragments of code. Keep in mind that you are showing DESIGN of your software, not the inner workings of it.
+
 
 +
'''SJ-One Board Monitoring Task'''
 +
 
 +
For this task, three tasks were implemented to gather data from the operating system, sensors, and pins of the SJ-One Board. The operating system task calculates the CPU usage of each running task in the system along with the amount of global memory used, allocated memory used, allocated memory available, and memory available on the system. The sensor task reads data from the temperature sensor, light sensor, and accelerometer on the board. The pin task checks if any of the on-board switches are being used or not. These three tasks run at low priority and sends all the data collected to a fourth task which is the consumer. The consumer receives the data, stores it in three separate queues, initializes the UART3 protocol on the board to prepare transmitting data to the Raspberry Pi 3 via the Xbee Wire Antenna module, and runs at a high priority. A fifth task called fresh clears each queue every 50 milliseconds to allow data to be sent to the consumer in real time. The fresh task runs at a medium priority and should always run at a lower priority level than the consumer.
 +
 
 +
 
 +
[[File:CMPE146 F16 MW MicroWatch Task Data flow Diagram.png | frame | center | <center> '''Figure 6. Task Data Flow Diagram''' </center>]]
 +
 
 +
 
 +
'''Raspberry Pi JSON Parsing and Firebase Upload'''
 +
 
 +
The python script utilizes several modules to read serial data from the Explorer dongle, parse the json string into an object, add a timestamp, and upload to firebase. The Python serial module is used to read from the Explorer dongle with a baud rate of 9600 and an infinite timeout. A loop to read the serial data is implemented using a try statement as occasionally incomplete data can be sent over the Xbee resulting in serial read errors, or failed json parsing. A secondary loop is used to ensure that the Python script halts until a complete line is received. This ensures that the json object can be parsed from the string and prevents data from piling up on the serial queue, delaying the latest information from being pushed to Firebase. Once the json string is parsed into a Python object, another Python module is used to add additional fields to the data for date and time. Once the data object is ready to be sent, it is serialized back into a string and attached to the body of an HTTP PUT request sent to Firebase's real-time database. Finally, the result of the HTTP PUT request is checked for the expected error code (200), and the uploaded json data is printed to the console.
 +
 
 +
<syntaxhighlight lang="python">
 +
while 1:
 +
    # Wait indefinitely for new serial data
 +
    while uartString is not json
 +
      uartString = readline(None)
 +
 
 +
    # add timestamp
 +
    uartJson['timestamp'] = time_hhmmss
 +
 
 +
    # update json in fireBase
 +
    result = requests.put(firebase_url, uartJson)
 +
    if result.status_code is 200:
 +
      print 'Data update Successful'
 +
    else:
 +
      print 'fireBase Data update failed'
 +
</syntaxhighlight>
 +
 
 +
 
 +
'''Amazon Alexa Skill'''
 +
 
 +
The Amazon Alexa skill development can be split up into two parts: the Alexa skill code hosted on Amazon Web Services (AWS) Lambda, and the Amazon Voice Services (AVS) Interaction model defined by text and json files. The Alexa skill was developed using the Alexa Skill Kit SDK for NodeJS and uses a series of intent handlers to process the incoming json files that hold user requests. For this application, several intents for user query, app launch, help, and exit are supported, each with their own function code. Upon receiving a user request for a query intent, the Alexa Skill code will look for the intended request, pull the desired data from Firebase, format it into a response, then send that response to AVS. The AVS interaction model is defined by the types of applicable intents found in the intent schema, the custom intent items found in the items list, and the sample utterances of the defined intents found in the utterances text. For example, if the user said, "What is the board's temperature," the utterance would be "''QueryIntent'' What is the board's {item}." The list of items would have "temperature" in it, and the intent schema would link the items list to the ''QueryIntent''.
 +
 
 +
 
 +
'''Intent Schema'''
 +
<syntaxhighlight lang="json">
 +
{
 +
  "intents": [
 +
    {
 +
      "intent": "QueryIntent",
 +
      "slots": [
 +
      {
 +
        "name" : "Item",
 +
        "type": "LIST_OF_ITEMS"
 +
      }
 +
    ]
 +
    },
 +
    {
 +
      "intent": "AMAZON.RepeatIntent"
 +
    },
 +
    {
 +
      "intent": "AMAZON.HelpIntent"
 +
    },
 +
    {
 +
      "intent": "AMAZON.StopIntent"
 +
    },
 +
    {
 +
      "intent": "AMAZON.CancelIntent"
 +
    }
 +
  ]
 +
}
 +
</syntaxhighlight>
 +
 
 +
 
 +
'''List of Items'''
 +
<syntaxhighlight lang="text">
 +
temp
 +
temperature
 +
light
 +
memory
 +
button
 +
accelerometer
 +
accel
 +
task
 +
tasks
 +
top tasks
 +
top three tasks
 +
cpu
 +
cpu usage
 +
task usage
 +
last update
 +
last update time
 +
</syntaxhighlight>
 +
 
 +
 
 +
'''Utterances'''
 +
<syntaxhighlight lang="text">
 +
QueryIntent what is the current {Item}
 +
QueryIntent what is the boards {Item}
 +
QueryIntent what is the board {Item}
 +
QueryIntent what is the {Item}
 +
QueryIntent tell me the current {Item}
 +
</syntaxhighlight>
  
 
=== Implementation ===
 
=== Implementation ===
This section includes implementation, but again, not the details, just the high level.  For example, you can list the steps it takes to communicate over a sensor, or the steps needed to write a page of memory onto SPI Flash.  You can include sub-sections for each of your component implementation.
+
 
 +
''' Transfer Data Object from the SJ-One Board to Raspberry Pi '''
 +
# Collect specified data
 +
# Format data as json in a character array
 +
# Initialize UART driver with a baud rate of 9600
 +
# Send null terminated character array over UART3
 +
 
 +
 
 +
''' Transfer Data Object from the Raspberry Pi to Firebase's Real-time Database'''
 +
# Initialize the serial object at "/dev/ttyUSB0" and continuously loop to read line
 +
# Receive string in python from serial connection to Explorer dongle
 +
# Parse json string to object and insert timestamp (date & time)
 +
# Serialize json object back to string
 +
# Send json string inside the body of an HTTP request to firebase
  
 
== Testing & Technical Challenges ==
 
== Testing & Technical Challenges ==
Describe the challenges of your project.  What advise would you give yourself or someone else if your project can be started from scratch again?
 
Make a smooth transition to testing section and described what it took to test your project.
 
  
Include sub-sections that list out a problem and solution, such as:
+
=== SJ-One Board Monitoring Task Issues ===
 +
While implementing the monitoring driver for the SJ-One Board, the queues that transmitted data to the consumer task were not functioning properly. Every compile run would net an issue with the queue.h header file that caused some confusion. To fix this issue, each section of the code was checked to see what section was creating the compile issues. It was discovered that our queues were sending the wrong data types to the consumer. Once the data types were modified to the proper characteristic of each queue, the issue was resolved.
  
=== My Issue #1 ===
+
 
Discuss the issue and resolution.
+
While implementing the Xbee driver within the consumer task, establishing the connection between the Xbee Wire Antenna on the SJ-One Board with the Xbee Wire Antenna on the Raspberry Pi 3 was troublesome. The first test was to see if data could be sent to the SJ-One Board from the Xbee Wire Antenna connected to the Digi XCTU software. The gets() function from the char_dev.hpp header file was implemented to see if data could be received and shown in Hercules. This function could not parse and display the test data sent as needed, so the next function implemented was the getChar() function from the char_dev.hpp header file. This function allowed data transmission to instantly be seen in Hercules as it was being typed in the XCTU console, which solved our receiving issue. Once this was complete, the second test ran was to send data out from the SJ-One Board. Using the putline() function from the char_dev.hpp header file, test data was sent to the XCTU console, the software successfully received it on the first try, and our sending and receiving issues were resolved.
 +
 
 +
=== Python Serial Read and JSON Parse Issues ===
 +
When initially designing the python script to read serial data and parse it into the desired HTTP PUT request, we created single loop to with a sleep timeout. In this implementation the loop would pull serial data from the boad's queue, send it, then go to sleep. This implementation had issue with timing as the rPi wake time now needed to be in sync with the SJ-one board to prevent a backlog of old data being retrieved from the rPi's serial queue. To improve data latency and reduce timing conflicts, we decided to abandon that system and instead have the python script parse and send monitoring data as soon as it was read from the Xbee. To do this, we created an additional loop to read a line of data from the serial connection and removed the sleep timer. Now that we had speedier and more stable script, other issues came into play as there was no way to know if the data received was fully intact. We found out that two issues could occur: the serial connection between the Explorer USB dongle and the Raspberry Pi 3 could temporarily freeze for a few millisecond, and the data that passes between the Xbees could sometimes be corrupt. Both of these cases caused a crash in the python script and needed to be carefully recovered from using 'try-except' statements. This process took some trial and error, but ultimately lead to huge improvements in running stability.
 +
 
 +
=== Alexa Skill Development Issues ===
 +
There are countless developer pitfalls and best practices when developing an Alexa skill, so here are some tips that might be most helpful to new developers.
 +
 
 +
* Become familiar with some basic Alexa Skill formats. There are really only a handful of Alexa Skill interaction models for useful skills. Take a look at the sample apps in the [https://github.com/alexa Official Amazon Alexa Github] for most of the basic types and try to adapt one to your needs.
 +
* Read and thoroughly understand how AVS interacts with the Lambda skill code. Some simple reading could have saved time from many hours of debugging
 +
* Setup a test environment with your skill code running locally. Some of the best tutorials for this is here: [https://developer.amazon.com/blogs/post/Tx24Z2QZP5RRTG1/new-alexa-technical-tutorial-debugging-aws-lambda-code-locally Debugging AWS Lambda Code Locally]
 +
* Test, test, and test again! You never know what the user is going to say, or how they will perceive (or misperceive) your skill, so be prepared for anything.
  
 
== Conclusion ==
 
== Conclusion ==
Conclude your project here. You can recap your testing and problems.  You should address the "so what" part here to indicate what you ultimately learnt from this project. How has this project increased your knowledge?
+
Designing this project is a great way to learn about the functionality of the SJ-One Board, Raspberry Pi 3, Xbee Wire Antenna, Google Firebase, and Amazon Alexa. Throughout testing, it was deduced that more time should be spent on debugging the project since the amount of time that was allocated for it was insufficient. If we were to recreate this project, we would expand the project's functionality to include the detection of the communication buses actively being used on the SJ-One Board. The Micro Watch Monitoring System is now fully functional and ready to be deployed to the future CMPE 146 classes!
  
 
=== Project Video ===
 
=== Project Video ===
Upload a video of your project and post the link here.
+
[https://youtu.be/0bk09lewxus YouTube]
  
 
=== Project Source Code ===
 
=== Project Source Code ===
[https://sourceforge.net/projects/sjsu/files/CmpE_S2016/ Sourceforge Source Code Link]
+
[https://github.com/MGItkin/FreeRTOS-distributed-monitoring Github Repository]
  
 
== References ==
 
== References ==
 
=== Acknowledgement ===
 
=== Acknowledgement ===
Any acknowledgement that you may wish to provide can be included here.
+
We would like to thank Preetpal Kang for the advice he has given us on this project and the curriculum of CMPE 146. This class is beneficial to our Computer Engineering profession as we have learned much about embedded system programming.
  
 
=== References Used ===
 
=== References Used ===
List any references used in project.
+
[http://www.socialledge.com/sjsu/index.php?title=SJ_One_Board SJ-One Board Info]
 +
 
 +
[https://www.raspberrypi.org/documentation/hardware/raspberrypi/schematics/RPI-3B-V1_2-SCHEMATIC-REDUCED.pdf Raspberry Pi 3 Datasheet]
 +
 
 +
[https://www.sparkfun.com/datasheets/Wireless/Zigbee/XBee-Datasheet.pdf Xbee Wire Antenna Datasheet]
 +
 
 +
[https://cdn.sparkfun.com/datasheets/Wireless/Zigbee/XBee-Explorer-Dongle-v23.pdf Xbee Explorer Dongle Datasheet]
 +
 
 +
[https://firebase.google.com/docs/ Google Firebase Documentation]
  
=== Appendix ===
+
[https://developer.amazon.com/alexa Amazon Alexa Documentation]
You can list the references you used.
 

Latest revision as of 23:43, 2 April 2017

Micro Watch Monitoring System

Abstract

The Micro Watch Monitoring System uses Amazon's Alexa Voice Service to relay monitored data from a network of SJ-One Boards with the assistance of the Raspberry Pi 3. The monitored data from a SJ-One Board is transmitted and received via a Xbee Wire Antenna module. The monitored data is then uploaded to our backend system, Google Firebase, by the Raspberry Pi 3, and is accessible through any device integrated with the Alexa Voice Service.

Objectives & Introduction

The objective of this project is to be able to monitor a network of SJ-One Boards for statistical data that is accessible via voice commands acknowledged by the Alexa Voice Service.

Tasks

  • Create FreeRTOS tasks to monitor sensor data, pin data, and OS statistics on the SJ-One Board
  • Create a task that parses sensor data into Javascript Object Notation (json) and utilizes the Xbee Wire Antenna to wirelessly transmit it via the UART protocol
  • Create a python script for the Raspberry Pi 3 that parses data received by the Xbee Wire Antenna and relays it to Firebase
  • Create an Alexa Skill that can answer user queries by accessing data in the Firebase Real-time Database.

Team Members & Responsibilities

Sanjay Agarwal

  • SJ-One Board Software Design
  • Hardware Connectivity Design
  • Software and Hardware System Testing

Mason Itkin

  • Raspberry Pi 3 Software Design
  • Amazon Alexa Voice Service Skill Design
  • Software and Voice Service System Testing

Schedule

Week Begin Task Task Description Result Planned Completion Actual Completion
1
10/21
Group Research on System Design
Complete
10/27
10/27
2
10/28
Design Group Task Structure
Complete
11/3
11/3
3
11/4
Order Raspberry Pi 3
Complete
11/10
11/10
4
11/11
Load NOOBs OS onto Raspberry Pi 3 and Deploy Alexa Skills on Amazon Web Services
Complete
11/17
11/17
5
11/18
Implement Alexa Voice Interactions and Communication to Raspberry Pi 3 via Lambda Function
Complete
11/24
11/24
6
11/25
Order Xbee Wire Antenna Modules and Xbee Explorer Dongle
Complete
12/1
12/1
7
12/2
Design SJ-One Board Monitoring Task and Xbee Driver
Complete
12/8
12/8
8
12/9
Design Raspberry Pi 3 Communication Protocol
Complete
12/15
12/15
9
12/16
Test Firmware, Hardware, and Voice Service Interconnectivity
Complete
12/17
12/19
10
12/21
Present Completed Project
Complete
12/21
12/21

Parts List & Cost

Part Quantity Price Merchant
SJ-One Board
1
$80
Preetpal Kang
Raspberry Pi 3
1
$35.70
Amazon
Raspberry Pi 3 NOOBs OS
1
$9.44
Amazon
Xbee Wire Antenna
2
$24.95
Amazon
Xbee Explorer Dongle
1
$26.95
Amazon

Design & Implementation

Hardware Interface

Micro Watch System Configuration

The Micro Watch system consists of the SJ-One Board, Raspberry Pi 3, Xbee Series 1 modules, and the Xbee Explorer Dongle.


Figure 1. Micro Watch System Configuration


Figure 2. Micro Watch System Configuration


Xbee 1mW Wire Antenna - Series 1

The Digi XCTU software is used to configure the Xbee Wire Antenna modules for wireless connection. A shared channel and Personal Area Network (PAN) are set for both of the modules as well as a destination and source address. Setting these variables is all that is needed to enable communication. The module that is placed on the SJ-One Board is setup as the coordinator while the module connected to the Raspberry Pi 3 via the Xbee Explorer Dongle is setup as an endpoint. Setting either modules would have the same effect in a Xbee network with two modules, but as this project is aimed at providing multiple SJ-One Boards with sensor reporting ability, this will be important later. The SJ-One Board sends data to the Raspberry Pi 3 via UART3 protocol, and the Raspberry Pi 3 parses this data upon receiving it.


Figure 3. Xbee Wire Antenna


Xbee Explorer Dongle

The Xbee Explorer Dongle is used to connect the Xbees to the Digi XCTU software as well as connect the Xbee to the Raspberry Pi 3. The Explorer dongle provides an interface between the Xbee pins and USB, allowing UART communication and configuration.


Figure 4. Xbee Explorer Dongle USB Controller


Figure 5. Xbee Explorer Dongle Pin Configuration

Software Design

SJ-One Board Monitoring Task

For this task, three tasks were implemented to gather data from the operating system, sensors, and pins of the SJ-One Board. The operating system task calculates the CPU usage of each running task in the system along with the amount of global memory used, allocated memory used, allocated memory available, and memory available on the system. The sensor task reads data from the temperature sensor, light sensor, and accelerometer on the board. The pin task checks if any of the on-board switches are being used or not. These three tasks run at low priority and sends all the data collected to a fourth task which is the consumer. The consumer receives the data, stores it in three separate queues, initializes the UART3 protocol on the board to prepare transmitting data to the Raspberry Pi 3 via the Xbee Wire Antenna module, and runs at a high priority. A fifth task called fresh clears each queue every 50 milliseconds to allow data to be sent to the consumer in real time. The fresh task runs at a medium priority and should always run at a lower priority level than the consumer.


Figure 6. Task Data Flow Diagram


Raspberry Pi JSON Parsing and Firebase Upload

The python script utilizes several modules to read serial data from the Explorer dongle, parse the json string into an object, add a timestamp, and upload to firebase. The Python serial module is used to read from the Explorer dongle with a baud rate of 9600 and an infinite timeout. A loop to read the serial data is implemented using a try statement as occasionally incomplete data can be sent over the Xbee resulting in serial read errors, or failed json parsing. A secondary loop is used to ensure that the Python script halts until a complete line is received. This ensures that the json object can be parsed from the string and prevents data from piling up on the serial queue, delaying the latest information from being pushed to Firebase. Once the json string is parsed into a Python object, another Python module is used to add additional fields to the data for date and time. Once the data object is ready to be sent, it is serialized back into a string and attached to the body of an HTTP PUT request sent to Firebase's real-time database. Finally, the result of the HTTP PUT request is checked for the expected error code (200), and the uploaded json data is printed to the console.

while 1:
    # Wait indefinitely for new serial data
    while uartString is not json
      uartString = readline(None)

    # add timestamp
    uartJson['timestamp'] = time_hhmmss
  
    # update json in fireBase
    result = requests.put(firebase_url, uartJson)
    if result.status_code is 200:
      print 'Data update Successful'
    else:
      print 'fireBase Data update failed'


Amazon Alexa Skill

The Amazon Alexa skill development can be split up into two parts: the Alexa skill code hosted on Amazon Web Services (AWS) Lambda, and the Amazon Voice Services (AVS) Interaction model defined by text and json files. The Alexa skill was developed using the Alexa Skill Kit SDK for NodeJS and uses a series of intent handlers to process the incoming json files that hold user requests. For this application, several intents for user query, app launch, help, and exit are supported, each with their own function code. Upon receiving a user request for a query intent, the Alexa Skill code will look for the intended request, pull the desired data from Firebase, format it into a response, then send that response to AVS. The AVS interaction model is defined by the types of applicable intents found in the intent schema, the custom intent items found in the items list, and the sample utterances of the defined intents found in the utterances text. For example, if the user said, "What is the board's temperature," the utterance would be "QueryIntent What is the board's {item}." The list of items would have "temperature" in it, and the intent schema would link the items list to the QueryIntent.


Intent Schema

{
  "intents": [
    {
      "intent": "QueryIntent",
      "slots": [
      {
        "name" : "Item",
        "type": "LIST_OF_ITEMS"
      }
     ]
    },
    {
      "intent": "AMAZON.RepeatIntent"
    },
    {
      "intent": "AMAZON.HelpIntent"
    },
    {
      "intent": "AMAZON.StopIntent"
    },
    {
      "intent": "AMAZON.CancelIntent"
    }
  ]
}


List of Items

temp
temperature
light
memory
button
accelerometer
accel
task
tasks
top tasks
top three tasks
cpu
cpu usage
task usage
last update
last update time


Utterances

QueryIntent what is the current {Item}
QueryIntent what is the boards {Item}
QueryIntent what is the board {Item}
QueryIntent what is the {Item}
QueryIntent tell me the current {Item}

Implementation

Transfer Data Object from the SJ-One Board to Raspberry Pi

  1. Collect specified data
  2. Format data as json in a character array
  3. Initialize UART driver with a baud rate of 9600
  4. Send null terminated character array over UART3


Transfer Data Object from the Raspberry Pi to Firebase's Real-time Database

  1. Initialize the serial object at "/dev/ttyUSB0" and continuously loop to read line
  2. Receive string in python from serial connection to Explorer dongle
  3. Parse json string to object and insert timestamp (date & time)
  4. Serialize json object back to string
  5. Send json string inside the body of an HTTP request to firebase

Testing & Technical Challenges

SJ-One Board Monitoring Task Issues

While implementing the monitoring driver for the SJ-One Board, the queues that transmitted data to the consumer task were not functioning properly. Every compile run would net an issue with the queue.h header file that caused some confusion. To fix this issue, each section of the code was checked to see what section was creating the compile issues. It was discovered that our queues were sending the wrong data types to the consumer. Once the data types were modified to the proper characteristic of each queue, the issue was resolved.


While implementing the Xbee driver within the consumer task, establishing the connection between the Xbee Wire Antenna on the SJ-One Board with the Xbee Wire Antenna on the Raspberry Pi 3 was troublesome. The first test was to see if data could be sent to the SJ-One Board from the Xbee Wire Antenna connected to the Digi XCTU software. The gets() function from the char_dev.hpp header file was implemented to see if data could be received and shown in Hercules. This function could not parse and display the test data sent as needed, so the next function implemented was the getChar() function from the char_dev.hpp header file. This function allowed data transmission to instantly be seen in Hercules as it was being typed in the XCTU console, which solved our receiving issue. Once this was complete, the second test ran was to send data out from the SJ-One Board. Using the putline() function from the char_dev.hpp header file, test data was sent to the XCTU console, the software successfully received it on the first try, and our sending and receiving issues were resolved.

Python Serial Read and JSON Parse Issues

When initially designing the python script to read serial data and parse it into the desired HTTP PUT request, we created single loop to with a sleep timeout. In this implementation the loop would pull serial data from the boad's queue, send it, then go to sleep. This implementation had issue with timing as the rPi wake time now needed to be in sync with the SJ-one board to prevent a backlog of old data being retrieved from the rPi's serial queue. To improve data latency and reduce timing conflicts, we decided to abandon that system and instead have the python script parse and send monitoring data as soon as it was read from the Xbee. To do this, we created an additional loop to read a line of data from the serial connection and removed the sleep timer. Now that we had speedier and more stable script, other issues came into play as there was no way to know if the data received was fully intact. We found out that two issues could occur: the serial connection between the Explorer USB dongle and the Raspberry Pi 3 could temporarily freeze for a few millisecond, and the data that passes between the Xbees could sometimes be corrupt. Both of these cases caused a crash in the python script and needed to be carefully recovered from using 'try-except' statements. This process took some trial and error, but ultimately lead to huge improvements in running stability.

Alexa Skill Development Issues

There are countless developer pitfalls and best practices when developing an Alexa skill, so here are some tips that might be most helpful to new developers.

  • Become familiar with some basic Alexa Skill formats. There are really only a handful of Alexa Skill interaction models for useful skills. Take a look at the sample apps in the Official Amazon Alexa Github for most of the basic types and try to adapt one to your needs.
  • Read and thoroughly understand how AVS interacts with the Lambda skill code. Some simple reading could have saved time from many hours of debugging
  • Setup a test environment with your skill code running locally. Some of the best tutorials for this is here: Debugging AWS Lambda Code Locally
  • Test, test, and test again! You never know what the user is going to say, or how they will perceive (or misperceive) your skill, so be prepared for anything.

Conclusion

Designing this project is a great way to learn about the functionality of the SJ-One Board, Raspberry Pi 3, Xbee Wire Antenna, Google Firebase, and Amazon Alexa. Throughout testing, it was deduced that more time should be spent on debugging the project since the amount of time that was allocated for it was insufficient. If we were to recreate this project, we would expand the project's functionality to include the detection of the communication buses actively being used on the SJ-One Board. The Micro Watch Monitoring System is now fully functional and ready to be deployed to the future CMPE 146 classes!

Project Video

YouTube

Project Source Code

Github Repository

References

Acknowledgement

We would like to thank Preetpal Kang for the advice he has given us on this project and the curriculum of CMPE 146. This class is beneficial to our Computer Engineering profession as we have learned much about embedded system programming.

References Used

SJ-One Board Info

Raspberry Pi 3 Datasheet

Xbee Wire Antenna Datasheet

Xbee Explorer Dongle Datasheet

Google Firebase Documentation

Amazon Alexa Documentation