HoloLens a new approach

In the past few years, Microsoft has made many changes. One of the products they invested in is the HoloLens (known as Project Baraboo), which brings a new era in technology. The HoloLens device is a pair of smartglass developed and built by Microsoft. It runs on Windows 10 operating system and has a set of sensors that bring cutting edge technology in your space. Currently, it’s a little expensive. The development edition costs $ 3,000 and the commercial suite for business costs $ 5000. As Microsoft announced, the new HoloLens is expected be released with a great reduction in price and a larger field of view in the first quarter of 2019.

Indicatively, some of the features of HoloLens are as follows:

CPU –     Intel 32-bit (1GHz)
Memory – 2 GB RAM, 1 GB HPU RAM
Storage – 64 GB (flash memory)
Display – 2.3 megapixel widescreen stereoscopic head-mounted display
Sound – Spatial sound technology
Input – Inertial measurement unit (Accelerometer, gyroscope, and magnetometer)
Camera – 2.4 MP
Weight – 579 g (1.28 lb)

My Holographic experience

I had the opportunity to work on HoloLens device in the last year and it was really great. It’s based on the use of two cutting edge technologies, spatial mapping and spatial sound. Spatial mapping merges the real with  the virtual world and as a result all the holograms seem real, while Spatial sound provides a 3D sound simulation experience using direction, distance and environment simulations. I can say that I have encountered some difficulties as HoloLens is a relatively new tool and many things had to be created from scratch, although the Microsoft  libraries from  Mixed Reality Toolkit (known as HoloToolkit)  were very helpful.

Read More »

In this tutorial I describe how to save and load data from text and XML files. The data I store is three-dimensional points into the mixed reality environment in which you can insert to list with gesture event air tap.

Add the GameObject “Camera”, Gaze and from the gestures air tap event which described in the guide “Gaze and Gestures on HoloLens“.

Create three GameObjects with Box Collider to use as buttons. With “Start Recording” tile you can start recording new three-dimensional points in the space with the use of air tap, with “Save Points” tile you can save the points in the text and XML file, with “Load Points” you can load the points from text and XML files. These GameObjects have a script for the OnSelect event.

OnSelectSave.cs file

    void OnSelect()
    {
        #if WINDOWS_UWP
        CRUDManager.Instance.SavePins();
        #endif
    }

Read More »

Microsoft HoloLens has a variety of sensors, making it a very powerful tool. The device is placed on the head and you can view the space from their point of view. This guide represents how you can take a photo via HoloLens and project it on a holographic surface and how you can record and play videos.

Initially, create a new scene with the name “Capture-Play-Photo-Video”. It is important to set the Camera of the scene, the projection to Perspective. Also, you need to be in the right position as shown in the picture. Position (0, -0.02, 0), Rotation (0, 0, 0), Scale (1, 1, 1). Most importantly, the point where you choose to display the camera is the point based on which the other objects in space will be initialized.

Add two new GameObjects (Cube) in the unity scene under the names “DisplayPhoto” and “PlayVideo” and give the following transform to the images.Read More »

In the last few years there has been great advancement in the field of neural networks and the Artificial Intelligence (AI). Microsoft has also developed a set of Cognitive Services that are a set on machine learning algorithms to solve problems in the field of Artificial Intelligence (AI). The Cognitive Services APIs are grouped into five categories:

• Vision—analyze images and videos for content and other useful information.
• Speech—tools to improve speech recognition and identify the speaker.
• Language—understanding sentences and intent rather than just words.
• Knowledge—tracks down research from scientific journals for you.
• Search—applies machine learning to web searches.

In this tutorial, we see how identification is used with the FACE API. There is also an additional GitHub guide for character recognition through the Computer Vision API (see the end of the article).

It is important to set in the Camera of the scene the projection to Perspective and to be in the right position as shown in the picture. Position (0, -0.02, 0), Rotation (0, 0, 0), Scale (1, 1, 1). Essentially the point where you choose to display the camera is the point based on which the other objects in space will be initialized.

You will need two GameObjects, the first GameObject named “PhotoCaptureManager” will contain the scripts and will be responsible for capturing the photo from the HoloLens device and then for sending to Azure Cognitive service to process it. The second GameObject named “AudioSource” will contain audio Source components in order to play the voice from text-to-speech implementation. For example “He is pretty much 26 years old and looks happy”.

Before you start coding, you need to create an Azure account and save the service key in order to use in your code. Choose the billing package. With F0 free package you can create 20 calls a minute and 30k a month.

Read More »

This guide includes two scenes. In the first scene, you have to calculate the distance between HoloLens and any surface from the real world and the second one is a calculation which summarizes the distance from multiple three-dimensional points.

The calculation of the distance between HoloLens and any three-dimensional point in the space is described by the following method. You can use the spatial mapping component from HoloToolkit in order to recognize the surfaces from the real-world environment. These two points are the location of the HoloLens and the position that HoloLens recognizes as a surface through Gaze.

GlobalCursor.cs

void Update()
    {
        var headPosition = Camera.main.transform.position;
        var gazeDirection = Camera.main.transform.forward;
        
        var gazePosition = GameObject.Find("GlobalCursor").transform.position;

        RaycastHit hitInfo;

        if (Physics.Raycast(headPosition, gazeDirection, out hitInfo))
        {
            meshRenderer.enabled = true;

            this.transform.position = hitInfo.point;
            
            this.transform.rotation = Quaternion.FromToRotation(Vector3.up, hitInfo.normal);

            //Display distance (Hololens - Cursor)
            float distance = Vector3.Distance(headPosition, gazePosition);
            TextMessage.Instance.ChangeTextMessage(string.Format("Distance: {0}", distance >= 1.0f ? distance.ToString("####0.0") + "m" : distance.ToString("0.00") + "cm" ));
        }
        else
        {
            meshRenderer.enabled = false;
            TextMessage.Instance.ChangeTextMessage(string.Empty);
        }
    }
}

Read More »

It is important to set in the Camera of the scene the projection to Perspective and to be in the right position as shown in the picture. Position (0, -0.02, 0), Rotation (0, 0, 0), Scale (1, 1, 1). Essentially, the point where you choose to display the camera is the point based on this which the other objects in space will be initialized.

Read More »