Throughout the years, we’ve seen Apple’s unwavering commitment to ensuring the accessibility of its devices for all users. Apple introduced several new accessibility features in iOS 17, among which is Point and Speak, a feature buried in the Magnifier app. Point and Speak helps people with visual disabilities to navigate and interact with their physical environment. Here’s everything you need to know on how to use the Point and Speak function on iPhone and iPad.
- What is Apple Point and Speak in Magnifier iOS 17?
- Pre-requisites to use Point and Speak on iPhone and iPad
- How to use Point and Speak in Magnifier app in iOS 17 on iPhone
- Benefits of using Point and Speak in Magnifier app
What is Apple Point and Speak in Magnifier iOS 17?
Point and Speak is a new accessibility feature built into Magnifier’s Detection Mode, which uses the iPhone or iPad’s sensors to detect nearby text, people, doors, and scenes and provide users with sound, speech, or haptic feedback.
Point and Speak combine on-device machine learning and the input from the device’s camera and LiDAR scanner to detect the position of your finger and read the text labels pointed at by your finger within the camera view.
Pre-requisites to use Point and Speak on iPhone and iPad
To use Point and Speak in the Magnifier app, your device needs to satisfy the following conditions:
- An iPhone or iPad with iOS 17 or iPadOS 17 or later.
- A device with a LiDAR sensor. LiDAR sensors are currently available on the following iPhone and iPad models:
- iPhone 15 Pro and 15 Pro Max
- iPhone 14 Pro and 14 Pro Max
- iPhone 13 Pro and 13 Pro Max
- iPhone 12 Pro and 12 Pro Max
- iPad Pro (2022)
- iPad Pro (2021)
- iPad Pro (2020)
Further, to use Point and Speak in the Magnifier app, you must first enable Detection Mode on Magnifier since Magnifier’s primary controls are for zooming in on things in the environment. Here’s how you can enable Detection Mode on iPhone.
How to enable Detection Mode in Magnifier app on iPhone
- Open the Magnifier app.
- Tap the gear icon → Settings. You’ll be directed to the Customize Controls screen.
- Locate Detection Mode. If a plus icon (+) is preceding it, drag it to either the primary or secondary controls to display the feature in the Magnifier. If you cannot add it, consider removing an existing control before replacing it with Detection Mode.
- Tap Done.
- If you’ve placed it under Primary Controls, it should be at the top level of icons in Magnifier.
- If you put it under Secondary Control, you can find its icon on the second layer. The Detection Mode icon looks like a square that looks like four brackets.
Once you have ensured all of the above, you can proceed towards using Point and Speak.
How to use Point and Speak in Magnifier app in iOS 17 on iPhone
- Open the Magnifier app on your iPhone or iPad.
- Select the Detection Mode icon (square-shaped).
- Now, tap the Point and Speak icon (the icon with a pointing finger). The icon will be highlighted in yellow once selected.
- Position your finger under the camera and wait for your device to announce that your finger has been detected. You’ll also see a Hand Detected notification on the screen.
- Keep your camera parallel to the surface or object being scanned.
- By default, Point and Speak will detect text under your finger. To change this, select Point and Speak → tap Settings on the upper-left side of the screen. Select Point and Speak. Under Location, tap Under or Over.
- Toggle Automatic Flashlight if you want Magnifier to automatically turn on the flashlight for 10 seconds when the environment is too dark to detect text.
You can also select which feedback you want to get whenever you use Point and Speak. Under Detection Feedback, toggle any of the following:
- Sound: You’ll hear a clicking sound whenever it detects something
- Labels: Your device will show the text it was able to recognize
- Speech: Your device will describe/read the text you’re pointing to
- Haptics: You’ll feel haptic feedback every time it identifies a text
By default, when you point to a text on your iPhone or iPad, it will be enclosed with a yellow border. However, you can modify the color of this border or remove it entirely if you prefer. Here’s how you can do it:
- To remove the border: Toggle the Border option to off.
- To change the border color: Tap Color. A range of color options will appear, allowing you to choose your preferred border color. You can Tap the Grid, Spectrum, or Sliders tab to explore and select the ideal color that suits your preference.
Benefits of using Point and Speak in Magnifier app
- Many individuals with low vision retain the ability to perceive outlines and distinguish high contrasts, allowing them to recognize buttons and objects in their environment, especially when marked with significant contrast. However, they may lack the visual acuity to decipher labels, particularly those with small fonts or tight spacing. In such cases, Point and Speak can prove immensely valuable for those requiring assistance interpreting printed labels on assorted objects.
- It can also be beneficial for people whose eyes get quickly tired. Point and Speak can read even the smallest labels before displaying the detected text in a bigger font and even narrate it for you, so you don’t have to strain your eyes to read tiny text.
FAQ
Magnifier is a built-in app on the iPhone and iPad. If you can’t find it, you may have deleted it. Go to the App Store to download it again.
Wrapping up…
Vision is a precious gift, enabling us to appreciate our surroundings and engage with the world. Unfortunately, many everyday objects have labels printed in small fonts, and for those experiencing a loss of visual clarity, this can hinder interaction with such objects.
Thankfully, innovative features like Point and Speak can transform your iPhone or iPad into a revolutionary magnifying tool that doesn’t just enlarge labels but also reads them aloud
Can you think of other ways Point and Speak can benefit its users? Share them below!
Read more:
